-- Logs begin at Sun 2022-10-23 00:10:42 UTC, end at Sun 2022-10-23 01:15:01 UTC. -- Oct 23 00:41:43.892582 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Oct 23 00:41:43.903330 dv-cinder-master-3-smb devstack@keystone.service[81360]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 23 00:41:43 2022] *** Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: nodename: dv-cinder-master-3-smb Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: machine: x86_64 Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: clock source: unix Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: pcre jit disabled Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: detected number of CPU cores: 4 Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: current working directory: / Oct 23 00:41:43.905394 dv-cinder-master-3-smb devstack@keystone.service[81360]: detected binary path: /usr/bin/uwsgi-core Oct 23 00:41:43.906025 dv-cinder-master-3-smb devstack@keystone.service[81360]: your processes number limit is 47680 Oct 23 00:41:43.906025 dv-cinder-master-3-smb devstack@keystone.service[81360]: your memory page size is 4096 bytes Oct 23 00:41:43.906025 dv-cinder-master-3-smb devstack@keystone.service[81360]: detected max file descriptor number: 2048 Oct 23 00:41:43.906025 dv-cinder-master-3-smb devstack@keystone.service[81360]: lock engine: pthread robust mutexes Oct 23 00:41:43.906025 dv-cinder-master-3-smb devstack@keystone.service[81360]: thunder lock: enabled Oct 23 00:41:43.906025 dv-cinder-master-3-smb devstack@keystone.service[81360]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 23 00:41:43.906025 dv-cinder-master-3-smb devstack@keystone.service[81360]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 23 00:41:43.945364 dv-cinder-master-3-smb devstack@keystone.service[81360]: Python main interpreter initialized at 0x55ce3a2d49c0 Oct 23 00:41:43.945364 dv-cinder-master-3-smb devstack@keystone.service[81360]: python threads support enabled Oct 23 00:41:43.945364 dv-cinder-master-3-smb devstack@keystone.service[81360]: your server socket listen backlog is limited to 100 connections Oct 23 00:41:43.945364 dv-cinder-master-3-smb devstack@keystone.service[81360]: your mercy for graceful operations on workers is 90 seconds Oct 23 00:41:43.945685 dv-cinder-master-3-smb devstack@keystone.service[81360]: mapped 403077 bytes (393 KB) for 2 cores Oct 23 00:41:43.945781 dv-cinder-master-3-smb devstack@keystone.service[81360]: *** Operational MODE: preforking *** Oct 23 00:41:43.946042 dv-cinder-master-3-smb devstack@keystone.service[81360]: *** uWSGI is running in multiple interpreter mode *** Oct 23 00:41:43.946042 dv-cinder-master-3-smb devstack@keystone.service[81360]: spawned uWSGI master process (pid: 81360) Oct 23 00:41:43.946219 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Oct 23 00:41:43.947755 dv-cinder-master-3-smb devstack@keystone.service[81360]: spawned uWSGI worker 1 (pid: 81371, cores: 1) Oct 23 00:41:43.947755 dv-cinder-master-3-smb devstack@keystone.service[81360]: spawned uWSGI worker 2 (pid: 81372, cores: 1) Oct 23 00:41:43.947755 dv-cinder-master-3-smb devstack@keystone.service[81360]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 23 00:41:48.184016 dv-cinder-master-3-smb devstack@keystone.service[81372]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 00:41:48.184016 dv-cinder-master-3-smb devstack@keystone.service[81372]: from cryptography.utils import int_from_bytes Oct 23 00:41:48.184801 dv-cinder-master-3-smb devstack@keystone.service[81372]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 00:41:48.184801 dv-cinder-master-3-smb devstack@keystone.service[81372]: from cryptography.utils import int_from_bytes Oct 23 00:41:48.184920 dv-cinder-master-3-smb devstack@keystone.service[81371]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 00:41:48.184920 dv-cinder-master-3-smb devstack@keystone.service[81371]: from cryptography.utils import int_from_bytes Oct 23 00:41:48.185333 dv-cinder-master-3-smb devstack@keystone.service[81371]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 00:41:48.185333 dv-cinder-master-3-smb devstack@keystone.service[81371]: from cryptography.utils import int_from_bytes Oct 23 00:41:48.234185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 23 00:41:48.234355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 23 00:41:48.234545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] command line args: [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 23 00:41:48.234715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 23 00:41:48.234796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 23 00:41:48.234912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 23 00:41:48.234997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ================================================================================ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 23 00:41:48.235107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] command line args: [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 23 00:41:48.235211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] admin_token = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.235379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 23 00:41:48.235459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config_dir = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.235558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ================================================================================ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 23 00:41:48.235643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.235791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] config_source = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.235868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] admin_token = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.235966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] debug = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.236045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config_dir = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.236162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.236501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.236501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] config_source = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.236711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.236807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] debug = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.236908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_config_append = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.237957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_dir = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_config_append = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_dir = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.238944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.239997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.240994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.241955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] publish_errors = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.242181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.242342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.242430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.242545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] publish_errors = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.242666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.242781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.242930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] standard_threads = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.243935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] standard_threads = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_journal = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_json = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_stderr = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] use_syslog = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_journal = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.244931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_json = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.245007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.245086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_stderr = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.245164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] use_syslog = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.245240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.245386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 00:41:48.245471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.245644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.245734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.245915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.246924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.247875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.248869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.249967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.250999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.251926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.252875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.253967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.254956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.255860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.256012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.256363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.256512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.256592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.256744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.256823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.256975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.257970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.258124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.258202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.258359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.258436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.259996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.260983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.261936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.262989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.263910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.264985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.265918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.266958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.267946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.268933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.269968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.270904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.external = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.271896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.password = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.external = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] auth.token = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.272975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.password = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] auth.token = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.273916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.274033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.274150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.274265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.274999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.275961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.276962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.277895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.278937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.279987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.280954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.281907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.282910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.283986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.284202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.284372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.284503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.284622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.284737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.284851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.284965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.285888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.286935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.287967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.288924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.289970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.290900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.291942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.292892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.293939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.294979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.295902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.296965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.297900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.298933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.299982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.300929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.301984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.302906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.303944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.304066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.304182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.304338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.304457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.304584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.304700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.304888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.305927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.306970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.307992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.308941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.309953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.310867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.311990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.312972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.313891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.314924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.315983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.316925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.317981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.318895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.319928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81372) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.320758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 00:41:48.322175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81371) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 23 00:41:48.348548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.348775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.348999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.349696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.349931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.350159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.357308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.358478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.385505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.385730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.386705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.386940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.438076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.438484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.486493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.486720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.487219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.487457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.643168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.643381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.643475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.643621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.643833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.644062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.644287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.644430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.644512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.644739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.644975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.645336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.667770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.667944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.668926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.669070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.669188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.680169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.681234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.693600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.693752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.693894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.693989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81372) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.694995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.695212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81371) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 00:41:48.712131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.712352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 00:41:48.712475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.712674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 00:41:48.713018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.713221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 00:41:48.713340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.713497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.713764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.713928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 00:41:48.714022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.714267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.714549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.714949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.715335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.715722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.723865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.724148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.724445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.724734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.725452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.725729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.725997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.726279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.731724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.733275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.734647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.734940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.735230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.735518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.735807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.736076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.736167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.736370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.736476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.736653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.736742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.736935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.737025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.737210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.737299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.737480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.737588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.737746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.738183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.738452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.738722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.738987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.761289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.762691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.762853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.763123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.764144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.764451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.767799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.768087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.768410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.769100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.769386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.769669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.774835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.775126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.776006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.776291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.781123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.781438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.781734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.782025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.782147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.782306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.782448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.782589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.782742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.782886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.783024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.783171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.783295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.783573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.783848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.784123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.798703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.798981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.799534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.799811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.802794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.803671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.806575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.806871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.807420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.807713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.813137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.813253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.814337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.814987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.817587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.817896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.818162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.818260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.818459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.818543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.818743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.818826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.819029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.819112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.819319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.819596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.834751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.835045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.835215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.835321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.835502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.835584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.835770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.836033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.842477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.842877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.844004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.844421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.845579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.845868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.845950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.846203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.848509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.848799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.848881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.849083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.849166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.849365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.849456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.849641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.849765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.850025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.861396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.861563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.861684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.861845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.861969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.862125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.862266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.862406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.862659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.862803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.862943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.863092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.863226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.863378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.879030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.879126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.882212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.882304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.885667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.885759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.887130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.887269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.888625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.888714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.888914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.888998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.889308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.889400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.895561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.895702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.897053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.897267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.900576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.900764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.900968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.901160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.901242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.901448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.901544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.901731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.909214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.909500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.909584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.909774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.909856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.910056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.916623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81372) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.916906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.917000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81371) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 00:41:48.917186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.917269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.917461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.917543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.917731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.917822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.918898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.919976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.920168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81372) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.920247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.920462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81371) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 00:41:48.954140 dv-cinder-master-3-smb devstack@keystone.service[81372]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ce3a2d49c0 pid: 81372 (default app) Oct 23 00:41:48.954488 dv-cinder-master-3-smb devstack@keystone.service[81371]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55ce3a2d49c0 pid: 81371 (default app) Oct 23 00:41:48.957325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a2c6-55c2-4833-8dae-3eea5cefba76 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:48.957609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a2c6-55c2-4833-8dae-3eea5cefba76 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:48.957877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182a2c6-55c2-4833-8dae-3eea5cefba76 None None] PATH_INFO: `/v3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:48.962645 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Sun Oct 23 00:41:48 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:41:55.709873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917627a0-dff9-4d3f-927b-d57853ae89a6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:55.710384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917627a0-dff9-4d3f-927b-d57853ae89a6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:55.710463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917627a0-dff9-4d3f-927b-d57853ae89a6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:55.714754 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:41:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:41:55.719956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:55.720444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:55.720587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:55.790550 dv-cinder-master-3-smb devstack@keystone.service[81372]: INFO dbcounter [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] Registered counter for database keystone Oct 23 00:41:55.809644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG oslo_db.sqlalchemy.engines [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81372) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 23 00:41:55.815969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writer thread running {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 23 00:41:55.993721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG passlib.handlers.bcrypt [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81372) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 23 00:41:55.994033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG passlib.handlers.bcrypt [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81372) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 23 00:41:56.051171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:41:56.052801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:41:56.179110 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:41:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 00:41:56.639583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917627a0-dff9-4d3f-927b-d57853ae89a6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:41:56.709469 dv-cinder-master-3-smb devstack@keystone.service[81371]: INFO dbcounter [None req-917627a0-dff9-4d3f-927b-d57853ae89a6 None None] Registered counter for database keystone Oct 23 00:41:56.731498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG oslo_db.sqlalchemy.engines [None req-917627a0-dff9-4d3f-927b-d57853ae89a6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81371) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 23 00:41:56.737705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writer thread running {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 23 00:41:56.855385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917627a0-dff9-4d3f-927b-d57853ae89a6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:41:56.859551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b989988-37c9-4c14-b974-61fd8c46bd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:41:56.860188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989988-37c9-4c14-b974-61fd8c46bd93 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:56.860501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989988-37c9-4c14-b974-61fd8c46bd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:56.860781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b989988-37c9-4c14-b974-61fd8c46bd93 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:56.868993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b989988-37c9-4c14-b974-61fd8c46bd93 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:41:56.920420 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9b989988-37c9-4c14-b974-61fd8c46bd93 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 23 00:41:56.920718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b989988-37c9-4c14-b974-61fd8c46bd93 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:41:56.929953 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Sun Oct 23 00:41:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:41:56.935847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:41:56.952210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a70c01-4461-4c43-820d-43841fdad7cd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:41:56.956561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebeb937-8ec2-495d-b333-ac0f082beacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:41:56.957653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb937-8ec2-495d-b333-ac0f082beacb admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:56.957928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb937-8ec2-495d-b333-ac0f082beacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:56.958195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb937-8ec2-495d-b333-ac0f082beacb admin admin] PATH_INFO: `/v3/projects/ee25af34c3554966b7a70b6f9a37af9a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:56.959785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeb937-8ec2-495d-b333-ac0f082beacb admin admin] RBAC: Authorizing `identity:get_project(project_id=ee25af34c3554966b7a70b6f9a37af9a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:41:57.013321 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ebeb937-8ec2-495d-b333-ac0f082beacb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 23 00:41:57.013658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeb937-8ec2-495d-b333-ac0f082beacb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:41:57.018093 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Sun Oct 23 00:41:56 2022] GET /identity/v3/projects/ee25af34c3554966b7a70b6f9a37af9a => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:41:59.125928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705592-276d-4fc7-8928-9639ef75754e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:59.126425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705592-276d-4fc7-8928-9639ef75754e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:59.126425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d705592-276d-4fc7-8928-9639ef75754e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:59.130482 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:41:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:41:59.136876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d6bb2-f03f-4bda-bf76-76e06b91abc2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:59.137307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d6bb2-f03f-4bda-bf76-76e06b91abc2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:59.137497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18d6bb2-f03f-4bda-bf76-76e06b91abc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:59.156886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f18d6bb2-f03f-4bda-bf76-76e06b91abc2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:41:59.158375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f18d6bb2-f03f-4bda-bf76-76e06b91abc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:41:59.165996 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:41:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 00:41:59.432988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d705592-276d-4fc7-8928-9639ef75754e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:41:59.444799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d705592-276d-4fc7-8928-9639ef75754e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:41:59.448929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eaeb82-6418-4143-ad6b-02fd812c4b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:41:59.449524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaeb82-6418-4143-ad6b-02fd812c4b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:59.449803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaeb82-6418-4143-ad6b-02fd812c4b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:59.450074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eaeb82-6418-4143-ad6b-02fd812c4b76 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:59.457588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaeb82-6418-4143-ad6b-02fd812c4b76 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:41:59.459330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eaeb82-6418-4143-ad6b-02fd812c4b76 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:41:59.467994 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Sun Oct 23 00:41:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:41:59.473074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d6bb2-f03f-4bda-bf76-76e06b91abc2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:41:59.482352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18d6bb2-f03f-4bda-bf76-76e06b91abc2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:41:59.486574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bae5c-41f7-4409-a55e-41fb38c68b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:41:59.487186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bae5c-41f7-4409-a55e-41fb38c68b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:41:59.487461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bae5c-41f7-4409-a55e-41fb38c68b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:41:59.487730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bae5c-41f7-4409-a55e-41fb38c68b31 admin admin] PATH_INFO: `/v3/users/c668a27e9bfc403386287995d4c4bda0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:41:59.489045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bae5c-41f7-4409-a55e-41fb38c68b31 admin admin] RBAC: Authorizing `identity:get_user(user_id=c668a27e9bfc403386287995d4c4bda0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:41:59.490870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bae5c-41f7-4409-a55e-41fb38c68b31 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:41:59.492373 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Sun Oct 23 00:41:59 2022] GET /identity/v3/users/c668a27e9bfc403386287995d4c4bda0 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:01.652270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def1a26-61a8-40b8-b0b9-cb15e4e781e6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:01.652830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def1a26-61a8-40b8-b0b9-cb15e4e781e6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:01.652830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def1a26-61a8-40b8-b0b9-cb15e4e781e6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:01.654196 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:42:01.659367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac1920-ec20-4fc6-afc9-1edd6980058c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:01.659796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac1920-ec20-4fc6-afc9-1edd6980058c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:01.660033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac1920-ec20-4fc6-afc9-1edd6980058c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:01.678071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-34ac1920-ec20-4fc6-afc9-1edd6980058c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:01.679238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-34ac1920-ec20-4fc6-afc9-1edd6980058c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:01.734646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b46f1-be0e-4fa0-b2ed-df6b3e6a8fee None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:01.734932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b46f1-be0e-4fa0-b2ed-df6b3e6a8fee None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:01.735210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b46f1-be0e-4fa0-b2ed-df6b3e6a8fee None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:01.736577 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:42:01.741711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:01.741996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:01.742267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:01.760906 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:01.783948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG passlib.handlers.bcrypt [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81371) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 23 00:42:01.784359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG passlib.handlers.bcrypt [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81371) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 23 00:42:01.834036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:01.835575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:01.933534 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:02.031404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac1920-ec20-4fc6-afc9-1edd6980058c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.056472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:02.137642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac1920-ec20-4fc6-afc9-1edd6980058c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.169644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.170279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.170562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.170838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.182145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:02.191034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:02.192086 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.193136 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:42:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:02.200560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.201383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.287483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:02.305535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46532eed-cb8f-4b8e-8b5a-4c4a84f3776b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.306218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b111c7-f22e-41c6-98c9-be8816ed2e85 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.308542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:02.341607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.342217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.342497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.342770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.343753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:02.343848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a677718-6873-484d-8971-b3828643735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.344690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a677718-6873-484d-8971-b3828643735f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.344973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a677718-6873-484d-8971-b3828643735f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.345188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:02.345278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a677718-6873-484d-8971-b3828643735f None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.346055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a677718-6873-484d-8971-b3828643735f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:02.346359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a677718-6873-484d-8971-b3828643735f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:02.347764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a677718-6873-484d-8971-b3828643735f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:02.355124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.sql.core [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 00:42:02.355124 dv-cinder-master-3-smb devstack@keystone.service[81372]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 00:42:02.355124 dv-cinder-master-3-smb devstack@keystone.service[81372]: [parameters: {'id': 'e3020de891fd49e3818a04aa3aebb8cd', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 00:42:02.355124 dv-cinder-master-3-smb devstack@keystone.service[81372]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81372) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 00:42:02.360075 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 8/15] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:02.365763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a677718-6873-484d-8971-b3828643735f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:42:02.372798 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 00:42:02.373758 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application yield resource Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise self.value Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application session.commit() Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 00:42:02.374698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.flush() Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application compat.raise_( Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 00:42:02.375519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 00:42:02.376455 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:42:02.377220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [parameters: {'id': 'e3020de891fd49e3818a04aa3aebb8cd', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 00:42:02.378069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:02.378794 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.380108 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 8/16] 10.222.0.39 () {60 vars in 1137 bytes} [Sun Oct 23 00:42:02 2022] POST /identity/v3/domains => generated 190 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:02.381084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.470115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a677718-6873-484d-8971-b3828643735f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.482185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f01da6-a0d7-415e-acd7-94c2157842fd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.503416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.504272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.504576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.504860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] PATH_INFO: `/v3/users/c668a27e9bfc403386287995d4c4bda0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.506601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] RBAC: Authorizing `identity:get_user(user_id=c668a27e9bfc403386287995d4c4bda0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:02.508047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:02.509651 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 9/17] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:02 2022] GET /identity/v3/users/c668a27e9bfc403386287995d4c4bda0 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:02.515108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.517549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.518174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.518450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.518718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.528850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:02.530304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:02.536945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:42:02.545897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:02.546766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:42:02.547701 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:42:02.547701 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:42:02.547701 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:02.547701 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 9/18] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:02.552055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:02.554270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.617839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79ff2f-0568-41e1-9e3d-02b4d0a5f85c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.651152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.651780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.652058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.652328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9b7a91-d6d4-43e2-b367-2d36045b2d24 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.652433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.653457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:02.654898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:02.656472 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 10/19] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:02.661970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:02.685901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae8720b-c2a5-4faa-9c27-336163c9c18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.686509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8720b-c2a5-4faa-9c27-336163c9c18c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.686790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8720b-c2a5-4faa-9c27-336163c9c18c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.687066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae8720b-c2a5-4faa-9c27-336163c9c18c None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.687736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8720b-c2a5-4faa-9c27-336163c9c18c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:02.688040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8720b-c2a5-4faa-9c27-336163c9c18c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:02.690417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae8720b-c2a5-4faa-9c27-336163c9c18c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:02.706937 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 10/20] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:42:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:02.762993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e13619-d6cb-45ca-933c-512901f35d99 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:02.764622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:02.799037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b365bd91-ab8e-4a02-bc21-f0f86f053891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:02.799708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365bd91-ab8e-4a02-bc21-f0f86f053891 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:02.799984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365bd91-ab8e-4a02-bc21-f0f86f053891 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:02.800257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b365bd91-ab8e-4a02-bc21-f0f86f053891 None admin] PATH_INFO: `/v3/domains/default/users/c668a27e9bfc403386287995d4c4bda0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:02.802215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365bd91-ab8e-4a02-bc21-f0f86f053891 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c668a27e9bfc403386287995d4c4bda0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:02.803692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b365bd91-ab8e-4a02-bc21-f0f86f053891 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:02.921088 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 11/21] 10.222.0.39 () {58 vars in 1361 bytes} [Sun Oct 23 00:42:02 2022] PUT /identity/v3/domains/default/users/c668a27e9bfc403386287995d4c4bda0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:42:05.448694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b4b4-6e61-43cc-9c41-cae816f7c459 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.449219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b4b4-6e61-43cc-9c41-cae816f7c459 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.449219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572b4b4-6e61-43cc-9c41-cae816f7c459 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:05.450872 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 11/22] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:05.457610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472c2c-6a42-4578-a836-bbac44072fe7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.458596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472c2c-6a42-4578-a836-bbac44072fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.459078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c472c2c-6a42-4578-a836-bbac44072fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:05.486355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1c472c2c-6a42-4578-a836-bbac44072fe7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:05.488201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1c472c2c-6a42-4578-a836-bbac44072fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:05.576351 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 12/23] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:05.669667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d625a13-9566-4471-9efc-f013346e5166 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.669979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d625a13-9566-4471-9efc-f013346e5166 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.670220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d625a13-9566-4471-9efc-f013346e5166 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:05.671842 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 12/24] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:05.676901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1046f-bb9b-48a5-82ac-62c06fd1e5e3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.677417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1046f-bb9b-48a5-82ac-62c06fd1e5e3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.677885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1046f-bb9b-48a5-82ac-62c06fd1e5e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:05.700345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7ad1046f-bb9b-48a5-82ac-62c06fd1e5e3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:05.701817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7ad1046f-bb9b-48a5-82ac-62c06fd1e5e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:05.786788 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 13/25] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:05.946959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38ebff-6b5a-454e-ae45-bc56698c9dbb None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.947237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38ebff-6b5a-454e-ae45-bc56698c9dbb None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.947760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38ebff-6b5a-454e-ae45-bc56698c9dbb None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:05.949338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 13/26] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:05.956056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8741c14-338d-4d8e-9fae-3f21c513fcfc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.956379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8741c14-338d-4d8e-9fae-3f21c513fcfc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.956665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8741c14-338d-4d8e-9fae-3f21c513fcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:05.960576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742fcdc-739c-4f21-b57f-15a426e082b0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.960854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742fcdc-739c-4f21-b57f-15a426e082b0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.961124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3742fcdc-739c-4f21-b57f-15a426e082b0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:05.962977 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 14/27] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:05.968767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da85cf-f7bc-47b1-bcf5-3b7372e7f1fa None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:05.969047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da85cf-f7bc-47b1-bcf5-3b7372e7f1fa None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:05.969322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da85cf-f7bc-47b1-bcf5-3b7372e7f1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:06.003611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f8741c14-338d-4d8e-9fae-3f21c513fcfc None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:06.004819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f8741c14-338d-4d8e-9fae-3f21c513fcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:06.007821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-19da85cf-f7bc-47b1-bcf5-3b7372e7f1fa None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:06.009067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-19da85cf-f7bc-47b1-bcf5-3b7372e7f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:06.038392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:06.098603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:06.112417 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 15/28] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:06.113424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da85cf-f7bc-47b1-bcf5-3b7372e7f1fa None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:06.143711 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 14/29] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:06.158451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8741c14-338d-4d8e-9fae-3f21c513fcfc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:06.214092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19da85cf-f7bc-47b1-bcf5-3b7372e7f1fa None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:06.249102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1a80b-7a4d-4f21-87ce-8bcde8d43070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:06.249985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a80b-7a4d-4f21-87ce-8bcde8d43070 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:06.250423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a80b-7a4d-4f21-87ce-8bcde8d43070 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:06.250829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1a80b-7a4d-4f21-87ce-8bcde8d43070 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:06.251630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1a80b-7a4d-4f21-87ce-8bcde8d43070 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:06.253248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc1a80b-7a4d-4f21-87ce-8bcde8d43070 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:06.343790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8741c14-338d-4d8e-9fae-3f21c513fcfc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:06.429834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507d549c-64ac-42e1-a501-917e8da6ad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:06.431283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d549c-64ac-42e1-a501-917e8da6ad66 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:06.431727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d549c-64ac-42e1-a501-917e8da6ad66 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:06.432151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507d549c-64ac-42e1-a501-917e8da6ad66 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:06.433011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d549c-64ac-42e1-a501-917e8da6ad66 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:06.434796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507d549c-64ac-42e1-a501-917e8da6ad66 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:06.465022 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 15/30] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Oct 23 00:42:06 2022] POST /identity/v3/roles => generated 225 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:06.489999 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 16/31] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Oct 23 00:42:06 2022] POST /identity/v3/roles => generated 219 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:06.520920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bccfb-f88a-47b3-8909-31d945ce897b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:06.521207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bccfb-f88a-47b3-8909-31d945ce897b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:06.521487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100bccfb-f88a-47b3-8909-31d945ce897b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:06.522852 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 16/32] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:06.529520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a952e1-533a-426a-8357-e22bd6859665 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:06.529843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a952e1-533a-426a-8357-e22bd6859665 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:06.530087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a952e1-533a-426a-8357-e22bd6859665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:06.552503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bccfb-f88a-47b3-8909-31d945ce897b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:06.557116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b8a952e1-533a-426a-8357-e22bd6859665 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:06.558756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b8a952e1-533a-426a-8357-e22bd6859665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:06.579547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:06.635956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:06.704577 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 17/33] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:06.712564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a952e1-533a-426a-8357-e22bd6859665 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:06.746347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100bccfb-f88a-47b3-8909-31d945ce897b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:06.782368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:06.783009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:06.783288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:06.783566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:06.804019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:06.805577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:42:06.831612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:06.832464 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:42:06.833250 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:42:06.833250 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:42:06.833250 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:06.833250 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 17/34] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:06.839453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:06.839917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a952e1-533a-426a-8357-e22bd6859665 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:06.892642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:06.893686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:06.893973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:06.894534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:06.908547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:06.915605 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:06.916866 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:06.917593 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:06.918631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:06.918739 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 18/35] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:42:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:06.925600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:06.973049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea8ebc-3760-4dfb-984c-0506f5ebe95b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:06.982272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:07.022540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.023391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.023818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.024225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.025669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:07.026131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.027661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.048648 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 18/36] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:07.054416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:07.066778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5d3a8-1f28-4445-92ea-348f8f916028 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:07.101958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.102614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.102904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.103173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.103856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:07.104164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.105830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.119688 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 19/37] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:07.120519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:07.174742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee32fa-d51d-4928-ab70-e43eeb076bdc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:07.200923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:07.213419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.214096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.214382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.214660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.215323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.216814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.220483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:07.224124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47062fa8-583d-4393-9174-8b7bf8d9ac0d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:07.248710 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 19/38] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:42:07 2022] POST /identity/v3/projects => generated 301 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:07.249549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:07.278487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53a6d73-75f5-4dcb-b4e0-6b35e633c866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.279509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6d73-75f5-4dcb-b4e0-6b35e633c866 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.279984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6d73-75f5-4dcb-b4e0-6b35e633c866 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.280476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53a6d73-75f5-4dcb-b4e0-6b35e633c866 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.281365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a6d73-75f5-4dcb-b4e0-6b35e633c866 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.283019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53a6d73-75f5-4dcb-b4e0-6b35e633c866 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.308931 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 20/39] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Oct 23 00:42:07 2022] POST /identity/v3/roles => generated 223 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:07.308931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ce9dc-71d4-46af-b835-06776dea5833 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.308931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ce9dc-71d4-46af-b835-06776dea5833 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.308931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ce9dc-71d4-46af-b835-06776dea5833 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.308931 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 21/40] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:07.314761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6710b7-e18d-4bef-baa5-8712cce29aee None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.315148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6710b7-e18d-4bef-baa5-8712cce29aee None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.315574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6710b7-e18d-4bef-baa5-8712cce29aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.381931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b83e0-cb8d-4727-b2d2-89f0ad2b4407 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:07.428808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.429834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.430314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.430759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] PATH_INFO: `/v3/users/c668a27e9bfc403386287995d4c4bda0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.432250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] RBAC: Authorizing `identity:get_user(user_id=c668a27e9bfc403386287995d4c4bda0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.434066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.441400 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 20/41] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:07 2022] GET /identity/v3/users/c668a27e9bfc403386287995d4c4bda0 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:07.442894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:07.451102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6b6710b7-e18d-4bef-baa5-8712cce29aee None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:07.452476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6b6710b7-e18d-4bef-baa5-8712cce29aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:07.500826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:07.552290 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 22/42] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:07.573216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe49454-8823-4801-af74-f954a41afec6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:07.608313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.608959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.609264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.609537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.610642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.612073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.613661 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 21/43] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:07.619397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6710b7-e18d-4bef-baa5-8712cce29aee None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:07.706363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:07.717687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6710b7-e18d-4bef-baa5-8712cce29aee None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:07.749789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.750395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.750676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.750950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.751673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.domain.id=default, user.id=c668a27e9bfc403386287995d4c4bda0) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:07.751980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.753442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.761139 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 23/44] 10.222.0.39 () {56 vars in 1322 bytes} [Sun Oct 23 00:42:07 2022] GET /identity/v3/role_assignments?user.id=c668a27e9bfc403386287995d4c4bda0&scope.domain.id=default&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:07.816785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:07.919406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3725322c-4a43-4f06-aae6-f4659a5d5ebe None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:07.932638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:07.954972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3f17ee-eadc-449e-9014-254270202aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:07.955608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f17ee-eadc-449e-9014-254270202aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:07.955880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f17ee-eadc-449e-9014-254270202aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:07.956150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3f17ee-eadc-449e-9014-254270202aa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:07.957272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f17ee-eadc-449e-9014-254270202aa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:07.958687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3f17ee-eadc-449e-9014-254270202aa7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:07.960231 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 22/45] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:07.965970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:08.065435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4d66ee-81b4-4fca-a003-9fb171459a6f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:08.097841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91b963-8740-48c3-9ed9-2d79a19c89a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:08.098384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91b963-8740-48c3-9ed9-2d79a19c89a3 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:08.098662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91b963-8740-48c3-9ed9-2d79a19c89a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:08.098937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91b963-8740-48c3-9ed9-2d79a19c89a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:08.099574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91b963-8740-48c3-9ed9-2d79a19c89a3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:08.101042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91b963-8740-48c3-9ed9-2d79a19c89a3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:08.126823 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 24/46] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:42:07 2022] POST /identity/v3/projects => generated 312 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:10.249307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d3e8d-d2c3-4a26-b9f3-f36544b807ed None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:10.249788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d3e8d-d2c3-4a26-b9f3-f36544b807ed None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:10.249788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6d3e8d-d2c3-4a26-b9f3-f36544b807ed None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:10.251111 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 23/47] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:10.256661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62b271-c5ad-4c6d-826c-28618f4391b3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:10.257152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62b271-c5ad-4c6d-826c-28618f4391b3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:10.257238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62b271-c5ad-4c6d-826c-28618f4391b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:10.270350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:10.275771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5a62b271-c5ad-4c6d-826c-28618f4391b3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:10.277198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5a62b271-c5ad-4c6d-826c-28618f4391b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:10.341852 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 25/48] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:10.609894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d3e8d-d2c3-4a26-b9f3-f36544b807ed None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:10.707756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6d3e8d-d2c3-4a26-b9f3-f36544b807ed None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:10.739756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d0e21d-1e3d-4b99-8ccc-37bf9541259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:10.740399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0e21d-1e3d-4b99-8ccc-37bf9541259e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:10.740681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0e21d-1e3d-4b99-8ccc-37bf9541259e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:10.740953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d0e21d-1e3d-4b99-8ccc-37bf9541259e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:10.742050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0e21d-1e3d-4b99-8ccc-37bf9541259e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:10.744081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d0e21d-1e3d-4b99-8ccc-37bf9541259e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:10.745636 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 24/49] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:10.751181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62b271-c5ad-4c6d-826c-28618f4391b3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:10.845288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62b271-c5ad-4c6d-826c-28618f4391b3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:10.878374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0b700-64c7-4b84-9d81-682988ce7740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:10.878924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b700-64c7-4b84-9d81-682988ce7740 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:10.879209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b700-64c7-4b84-9d81-682988ce7740 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:10.879484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0b700-64c7-4b84-9d81-682988ce7740 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:10.880128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b700-64c7-4b84-9d81-682988ce7740 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:10.881865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0b700-64c7-4b84-9d81-682988ce7740 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:10.887537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:10.907914 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 26/50] 10.222.0.39 () {60 vars in 1140 bytes} [Sun Oct 23 00:42:10 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:13.009319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce377a-21ef-4938-bcf1-53e3e086e82b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:13.009845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce377a-21ef-4938-bcf1-53e3e086e82b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:13.009845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce377a-21ef-4938-bcf1-53e3e086e82b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:13.011207 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 25/51] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:13.016278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23e48e-7d4f-446c-8b8f-c510831a6dd9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:13.016653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23e48e-7d4f-446c-8b8f-c510831a6dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:13.016855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23e48e-7d4f-446c-8b8f-c510831a6dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:13.034532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-aa23e48e-7d4f-446c-8b8f-c510831a6dd9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:13.035717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-aa23e48e-7d4f-446c-8b8f-c510831a6dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:13.099569 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 27/52] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:13.364364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce377a-21ef-4938-bcf1-53e3e086e82b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:13.412202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:13.462244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce377a-21ef-4938-bcf1-53e3e086e82b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:13.494735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d3674-9713-4536-8926-ba2366073817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:13.495352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d3674-9713-4536-8926-ba2366073817 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:13.495631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d3674-9713-4536-8926-ba2366073817 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:13.495901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d3674-9713-4536-8926-ba2366073817 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:13.497004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d3674-9713-4536-8926-ba2366073817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:13.498417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d3674-9713-4536-8926-ba2366073817 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:13.499984 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 26/53] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:13.505496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23e48e-7d4f-446c-8b8f-c510831a6dd9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:13.600519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23e48e-7d4f-446c-8b8f-c510831a6dd9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:13.616687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:13.633996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffafc04-1ce5-4b96-a82d-7dbb180d228e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:13.634600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffafc04-1ce5-4b96-a82d-7dbb180d228e None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:13.634878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffafc04-1ce5-4b96-a82d-7dbb180d228e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:13.635150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffafc04-1ce5-4b96-a82d-7dbb180d228e None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:13.635785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffafc04-1ce5-4b96-a82d-7dbb180d228e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:13.637262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffafc04-1ce5-4b96-a82d-7dbb180d228e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:13.662565 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 28/54] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:42:13 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:16.962909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7807-4ea1-4adc-a1ff-83b2ad691143 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:16.963975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7807-4ea1-4adc-a1ff-83b2ad691143 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:16.964521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc7807-4ea1-4adc-a1ff-83b2ad691143 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:16.966276 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 27/55] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:16.978474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df1b00-18a4-4399-942a-faf58cd84d4d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:16.979452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df1b00-18a4-4399-942a-faf58cd84d4d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:16.979925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df1b00-18a4-4399-942a-faf58cd84d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.009972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a7df1b00-18a4-4399-942a-faf58cd84d4d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:17.011511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a7df1b00-18a4-4399-942a-faf58cd84d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:17.100051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1025-62a5-4c3d-a213-63c0c0075616 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.100711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1025-62a5-4c3d-a213-63c0c0075616 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.100834 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 29/56] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:17.101329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416b1025-62a5-4c3d-a213-63c0c0075616 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.103252 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 28/57] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:17.108909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbbbf4-fccf-4c2f-a088-8a6145e6f52c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.109199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbbbf4-fccf-4c2f-a088-8a6145e6f52c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.109480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cbbbf4-fccf-4c2f-a088-8a6145e6f52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.137492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-54cbbbf4-fccf-4c2f-a088-8a6145e6f52c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:17.138720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-54cbbbf4-fccf-4c2f-a088-8a6145e6f52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:17.251921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26221-3c3f-43b0-a2e7-746667c15c69 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.252582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26221-3c3f-43b0-a2e7-746667c15c69 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.253082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f26221-3c3f-43b0-a2e7-746667c15c69 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.253208 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 30/58] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:17.255325 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 29/59] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:17.270985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e0104-0136-48cd-b7c3-cd37d2b4bc2a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.271298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e0104-0136-48cd-b7c3-cd37d2b4bc2a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.271548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e0104-0136-48cd-b7c3-cd37d2b4bc2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.305882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-de1e0104-0136-48cd-b7c3-cd37d2b4bc2a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:17.307107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-de1e0104-0136-48cd-b7c3-cd37d2b4bc2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:17.345349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:17.360505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786ee6b-b6b3-41c9-b6c5-8d431e9ce09e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.360798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786ee6b-b6b3-41c9-b6c5-8d431e9ce09e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.360988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786ee6b-b6b3-41c9-b6c5-8d431e9ce09e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.362588 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 30/60] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:17.367197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d2214-9e05-46f4-ba82-aaccb44a4de8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.368340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d2214-9e05-46f4-ba82-aaccb44a4de8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.368817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d2214-9e05-46f4-ba82-aaccb44a4de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.395395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-440d2214-9e05-46f4-ba82-aaccb44a4de8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:17.399627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-440d2214-9e05-46f4-ba82-aaccb44a4de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:17.428026 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 31/61] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:17.463374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e0104-0136-48cd-b7c3-cd37d2b4bc2a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:17.521032 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 31/62] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:17.580757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d2214-9e05-46f4-ba82-aaccb44a4de8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:17.628784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e0104-0136-48cd-b7c3-cd37d2b4bc2a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:17.630768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:17.675834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227aa1aa-7ca1-4dfa-ae8d-16918b9fc303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:17.676954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa1aa-7ca1-4dfa-ae8d-16918b9fc303 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.677743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa1aa-7ca1-4dfa-ae8d-16918b9fc303 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.678196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227aa1aa-7ca1-4dfa-ae8d-16918b9fc303 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.693125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227aa1aa-7ca1-4dfa-ae8d-16918b9fc303 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:17.695229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d2214-9e05-46f4-ba82-aaccb44a4de8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-227aa1aa-7ca1-4dfa-ae8d-16918b9fc303 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:17.702233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:17.703067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:17.703809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:17.705667 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 32/63] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:17.707137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bec81-9a2c-4f7f-879a-383fdac98785 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.707425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bec81-9a2c-4f7f-879a-383fdac98785 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.707697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2bec81-9a2c-4f7f-879a-383fdac98785 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.708955 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 33/64] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:17.720070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889a40-2a33-4d52-8914-6146d8aa403d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.720070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889a40-2a33-4d52-8914-6146d8aa403d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.720070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889a40-2a33-4d52-8914-6146d8aa403d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.736363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8a889a40-2a33-4d52-8914-6146d8aa403d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:17.737259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:17.737977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.738162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8a889a40-2a33-4d52-8914-6146d8aa403d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:17.738257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.738530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.749890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:17.756566 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:17.757398 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:17.758140 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 32/65] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:42:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:17.758802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:17.764282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:17.822390 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 34/66] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:17.823379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a889a40-2a33-4d52-8914-6146d8aa403d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:17.938357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a889a40-2a33-4d52-8914-6146d8aa403d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:17.975222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:17.975867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:17.976153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:17.976464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:17.987510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:17.990858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f5c0f-c123-4e33-a610-bb1cc3f5f526 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:17.992627 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:17.993977 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:17.994714 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 35/67] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:42:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:17.996502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.006712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.015412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.036506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.037108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.037590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.038051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.038735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:18.039040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.040444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.048777 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 33/68] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.049641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.112630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717adc35-2dd3-4b53-9da0-7299b250c4fb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.145660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.146449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.146909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.147186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.147850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:18.148156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.149344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47276663-baad-4172-ba41-d7437f8b4836 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.149602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.157289 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 36/69] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:42:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.158531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.186888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.187516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.187801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.188077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.197997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:18.202714 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:18.203594 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:18.204369 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 34/70] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:18.204988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.231728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.236628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.260356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ff2bf-7d96-4fd1-96e3-de2693582cff None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.292652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.293253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.293538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.293810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.294473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:18.294774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.296155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.303637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9ffb64-bcde-4213-9719-2cd53a4e0557 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.303971 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 37/71] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.304797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.339284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.339915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.340200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.340505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] PATH_INFO: `/v3/users/0b5a204a6bc6400198969e6cf752082a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.355744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] RBAC: Authorizing `identity:get_user(user_id=0b5a204a6bc6400198969e6cf752082a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.357201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.358823 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 35/72] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/users/0b5a204a6bc6400198969e6cf752082a => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.359577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.412256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17932f69-03d0-4f9b-9afa-77b88a1c1e51 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.441036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.446529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.447170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.447463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.447739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.448943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.450333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.452058 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 38/73] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.452683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.455979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef893584-3630-45a4-b116-5582d53fa9c9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.465139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.490365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.490996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.491279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.491552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] PATH_INFO: `/v3/users/0b5a204a6bc6400198969e6cf752082a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.492790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] RBAC: Authorizing `identity:get_user(user_id=0b5a204a6bc6400198969e6cf752082a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.495011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.496800 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 36/74] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/users/0b5a204a6bc6400198969e6cf752082a => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.497404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.557293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc084f16-601f-42be-afef-31e1dac22f66 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.590537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.591139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.591422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.591700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.592391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:18.592697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.594062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.595679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de353a-c5a7-42f5-a28f-ffb907ed15ed None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.601888 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 39/75] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.603204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.630927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.631823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.632104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.632417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] PATH_INFO: `/v3/users/c668a27e9bfc403386287995d4c4bda0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.633588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] RBAC: Authorizing `identity:get_user(user_id=c668a27e9bfc403386287995d4c4bda0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.635025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.636635 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 37/76] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/users/c668a27e9bfc403386287995d4c4bda0 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.640455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.662122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.671981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.704579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eddff9-3b6d-4b2c-9c3a-a58b988fb95e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.736662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58356d22-1e18-4311-901c-1c89eff39fdc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.737623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.738259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.738541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.738827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.739962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.741458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.743088 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 40/77] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.743848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.770307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.770916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.771195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.771466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.772111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.773563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.801891 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 38/78] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:42:18 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:18.802766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.857481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a86f5c8-ebc6-4f55-b060-a875a05233ac None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.895262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.896277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.896599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.896875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.898069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.900443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.902304 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 41/79] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.902937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:18.908364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.910697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:18.938000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6840b0b9-e011-4830-a358-dbbd98230814 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:18.978508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:18.979076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:18.979362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:18.979641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] PATH_INFO: `/v3/users/0b5a204a6bc6400198969e6cf752082a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:18.981299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] RBAC: Authorizing `identity:get_user(user_id=0b5a204a6bc6400198969e6cf752082a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:18.982789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:18.984475 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 39/80] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/users/0b5a204a6bc6400198969e6cf752082a => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:18.985242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:19.017352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c446e-da32-4f10-b4fc-e5d06e364afa None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:19.058279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:19.059250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:19.059534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:19.059942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:19.061037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:19.062504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:19.064332 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 42/81] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:18 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:19.064937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:19.103994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61828e-5f01-4486-be42-32ef4057e79e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:19.147036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:19.147718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:19.147991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:19.148272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/0b5a204a6bc6400198969e6cf752082a/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:19.150571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:19.150826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] RBAC: Authorizing `identity:create_grant(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76, user_id=0b5a204a6bc6400198969e6cf752082a, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:19.152282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:19.172442 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 40/82] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:18 2022] PUT /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/0b5a204a6bc6400198969e6cf752082a/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:19.173810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:19.185233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:19.187933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b38c2e-6061-4e40-b708-0e6c37adf648 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:19.243934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:19.244609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:19.244894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:19.246445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/0b5a204a6bc6400198969e6cf752082a/roles/633fd4ad499a4f62a5a2cbac0eb861ce` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:19.250790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] RBAC: Authorizing `identity:create_grant(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76, user_id=0b5a204a6bc6400198969e6cf752082a, role_id=633fd4ad499a4f62a5a2cbac0eb861ce)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:19.254222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:19.288748 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 43/83] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:19 2022] PUT /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/0b5a204a6bc6400198969e6cf752082a/roles/633fd4ad499a4f62a5a2cbac0eb861ce => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:19.290570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:19.386454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160223a6-6e8d-4319-8861-608a73467872 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:19.426493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:19.427157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:19.427439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:19.427715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] PATH_INFO: `/v3/projects/2e28229c023a440a886354a80918e350` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:19.429025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] RBAC: Authorizing `identity:get_project(project_id=2e28229c023a440a886354a80918e350)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:19.431175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:19.432928 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 41/84] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:19 2022] GET /identity/v3/projects/2e28229c023a440a886354a80918e350 => generated 312 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:19.438612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:19.463053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f395560-086c-403e-8e02-f58865b0d03b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:19.498731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:19.505519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910015a2-6699-4571-910b-baff1fb3a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:19.506744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910015a2-6699-4571-910b-baff1fb3a8bb None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:19.507340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910015a2-6699-4571-910b-baff1fb3a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:19.508084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910015a2-6699-4571-910b-baff1fb3a8bb None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/c668a27e9bfc403386287995d4c4bda0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:19.510946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910015a2-6699-4571-910b-baff1fb3a8bb None admin] RBAC: Authorizing `identity:create_grant(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76, user_id=c668a27e9bfc403386287995d4c4bda0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:19.513600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910015a2-6699-4571-910b-baff1fb3a8bb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:19.543249 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 44/85] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:19 2022] PUT /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/c668a27e9bfc403386287995d4c4bda0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:19.548732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:19.559577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786ae2-443c-4c9a-9e85-d81ef4f62399 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:19.599612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ead2e-4f4f-4316-b014-34a133a58205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:19.600327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ead2e-4f4f-4316-b014-34a133a58205 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:19.600636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ead2e-4f4f-4316-b014-34a133a58205 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:19.601172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ead2e-4f4f-4316-b014-34a133a58205 None admin] PATH_INFO: `/v3/projects/2e28229c023a440a886354a80918e350/users/0b5a204a6bc6400198969e6cf752082a/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:19.603773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ead2e-4f4f-4316-b014-34a133a58205 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e28229c023a440a886354a80918e350, user_id=0b5a204a6bc6400198969e6cf752082a, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:19.605292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ead2e-4f4f-4316-b014-34a133a58205 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:19.622260 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 42/86] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:19 2022] PUT /identity/v3/projects/2e28229c023a440a886354a80918e350/users/0b5a204a6bc6400198969e6cf752082a/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:21.833263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1e746-e1b0-4dc9-a7b9-b9692f3d3a32 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:21.833773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1e746-e1b0-4dc9-a7b9-b9692f3d3a32 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:21.833843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1e746-e1b0-4dc9-a7b9-b9692f3d3a32 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:21.835380 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 45/87] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:21.854430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f94ce0-e7d0-4fb1-903a-d23c4b71987f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:21.854944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f94ce0-e7d0-4fb1-903a-d23c4b71987f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:21.855014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f94ce0-e7d0-4fb1-903a-d23c4b71987f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:21.886465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c1f94ce0-e7d0-4fb1-903a-d23c4b71987f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:21.887636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c1f94ce0-e7d0-4fb1-903a-d23c4b71987f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:21.965308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec314-113a-4269-a313-267cb06f1983 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:21.965491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec314-113a-4269-a313-267cb06f1983 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:21.965738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ec314-113a-4269-a313-267cb06f1983 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:21.967335 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 46/88] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:42:21.972964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22427f9-a768-4c8a-9d19-51907ee9d97f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:21.973275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22427f9-a768-4c8a-9d19-51907ee9d97f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:21.973514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22427f9-a768-4c8a-9d19-51907ee9d97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.004157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c22427f9-a768-4c8a-9d19-51907ee9d97f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:22.005410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c22427f9-a768-4c8a-9d19-51907ee9d97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:22.013327 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 43/89] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:22.100078 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 47/90] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:22.144913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d1ced-3ac4-48de-b05c-275c6cc25c14 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:22.145212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d1ced-3ac4-48de-b05c-275c6cc25c14 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:22.145466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d1ced-3ac4-48de-b05c-275c6cc25c14 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.146962 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 44/91] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:22.153012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91827f1-7d81-4918-996a-3f0354b9be8b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:22.153301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91827f1-7d81-4918-996a-3f0354b9be8b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:22.153579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91827f1-7d81-4918-996a-3f0354b9be8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.176860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e91827f1-7d81-4918-996a-3f0354b9be8b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:22.178028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e91827f1-7d81-4918-996a-3f0354b9be8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:22.265756 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 48/92] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:22.442975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d1ced-3ac4-48de-b05c-275c6cc25c14 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:22.491066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91827f1-7d81-4918-996a-3f0354b9be8b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:22.540702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:22.590930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:22.612897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d1ced-3ac4-48de-b05c-275c6cc25c14 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:22.673210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91827f1-7d81-4918-996a-3f0354b9be8b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:22.683390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:22.684108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:22.684409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:22.684678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.702222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:22.708619 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:22.709533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:22.710269 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 45/93] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:22.714380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:22.745061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:22.746148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:22.747404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:22.752707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.757081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:22.759284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:22.760891 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 49/94] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:22.761497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:22.848013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2f6d0-3406-4a15-b559-27961505a68b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:22.877013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:22.892686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:22.893625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:22.894104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:22.894570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.908954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:22.915459 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:22.916274 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:22.917063 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:22.917618 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 46/95] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:22.918728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:22.932593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a680c60-b81f-42e8-b793-ee4af3111cb2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:22.954778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:22.973201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868e8ed9-5ed8-42f1-ac2b-77fd69fe4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:22.974644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e8ed9-5ed8-42f1-ac2b-77fd69fe4ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:22.974943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e8ed9-5ed8-42f1-ac2b-77fd69fe4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:22.975222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868e8ed9-5ed8-42f1-ac2b-77fd69fe4ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.975904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e8ed9-5ed8-42f1-ac2b-77fd69fe4ee2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:22.976213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e8ed9-5ed8-42f1-ac2b-77fd69fe4ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:22.977728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868e8ed9-5ed8-42f1-ac2b-77fd69fe4ee2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:22.989433 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 50/96] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:22.991120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0be514-e76a-4b63-9728-a11966574a80 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:22.991568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0be514-e76a-4b63-9728-a11966574a80 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:22.991853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0be514-e76a-4b63-9728-a11966574a80 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:22.994002 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 51/97] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:22.999211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0be514-e76a-4b63-9728-a11966574a80 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:23.055648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9138fc9-790b-4a6d-8f7d-4a959e51971c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:23.093854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:23.094485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.094771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.095051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.095727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:23.098040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:23.128040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0be514-e76a-4b63-9728-a11966574a80 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:23.132445 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 47/98] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:42:22 2022] POST /identity/v3/projects => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:23.134148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:23.182154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:23.184829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be980b-2be8-4207-aa0e-4afec8805501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:23.185720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be980b-2be8-4207-aa0e-4afec8805501 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.186201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be980b-2be8-4207-aa0e-4afec8805501 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.186712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be980b-2be8-4207-aa0e-4afec8805501 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.187665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be980b-2be8-4207-aa0e-4afec8805501 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:23.188169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be980b-2be8-4207-aa0e-4afec8805501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:23.191233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be980b-2be8-4207-aa0e-4afec8805501 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:23.206901 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 52/99] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:23.206901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fce12-e598-4f8d-9e8d-43af6180e118 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.206901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fce12-e598-4f8d-9e8d-43af6180e118 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.206901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7fce12-e598-4f8d-9e8d-43af6180e118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.222972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2a7fce12-e598-4f8d-9e8d-43af6180e118 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:23.224274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2a7fce12-e598-4f8d-9e8d-43af6180e118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:23.264857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e471ae-f371-439f-b434-64223ed7601e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:23.269280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:23.300128 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 53/100] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:23.301510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3628fcd-db50-4dee-8644-e75bfb68c68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:23.301631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763fa1-4e4e-4d62-8eba-0a9159c7c149 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.301828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763fa1-4e4e-4d62-8eba-0a9159c7c149 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.302109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6763fa1-4e4e-4d62-8eba-0a9159c7c149 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.302203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628fcd-db50-4dee-8644-e75bfb68c68a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.302426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628fcd-db50-4dee-8644-e75bfb68c68a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.302701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3628fcd-db50-4dee-8644-e75bfb68c68a None admin] PATH_INFO: `/v3/users/ec0fa36db19a4b8ba2755d8066da3c95` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.303426 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 54/101] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:23.309347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961a522-5205-4be3-8a1b-0a5e3312636c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.309630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961a522-5205-4be3-8a1b-0a5e3312636c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.309903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961a522-5205-4be3-8a1b-0a5e3312636c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.311401 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 55/102] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:23.311959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961a522-5205-4be3-8a1b-0a5e3312636c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:23.326913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3628fcd-db50-4dee-8644-e75bfb68c68a None admin] RBAC: Authorizing `identity:get_user(user_id=ec0fa36db19a4b8ba2755d8066da3c95)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:23.328448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3628fcd-db50-4dee-8644-e75bfb68c68a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:23.330522 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 48/103] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:23 2022] GET /identity/v3/users/ec0fa36db19a4b8ba2755d8066da3c95 => generated 289 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:23.335894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4091e-1032-4a59-88f3-bc69425740f6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.336021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4091e-1032-4a59-88f3-bc69425740f6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.336351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4091e-1032-4a59-88f3-bc69425740f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.355527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-71e4091e-1032-4a59-88f3-bc69425740f6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:23.356747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-71e4091e-1032-4a59-88f3-bc69425740f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:23.438883 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 49/104] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:23.440131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81989-f444-4a24-b57e-9772012f3189 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.440493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81989-f444-4a24-b57e-9772012f3189 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.440771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e81989-f444-4a24-b57e-9772012f3189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.445776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961a522-5205-4be3-8a1b-0a5e3312636c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:23.463492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f7e81989-f444-4a24-b57e-9772012f3189 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:23.465174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7e81989-f444-4a24-b57e-9772012f3189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:23.481799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:23.482517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.482746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.483126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] PATH_INFO: `/v3/users/0b5a204a6bc6400198969e6cf752082a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.484197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] RBAC: Authorizing `identity:get_user(user_id=0b5a204a6bc6400198969e6cf752082a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:23.485772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:23.485893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:23.487842 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 56/105] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:23 2022] GET /identity/v3/users/0b5a204a6bc6400198969e6cf752082a => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:23.492401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:23.549799 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 50/106] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:23.550973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e81989-f444-4a24-b57e-9772012f3189 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:23.556357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:23.673604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fd049d-afb7-4119-997c-c23bab15f800 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:23.715999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:23.717948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.719681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.720150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.722359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:23.724973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:23.729372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e81989-f444-4a24-b57e-9772012f3189 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:23.730857 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 57/107] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:23 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:23.733174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:23.799370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:23.801060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.801340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.801622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.838416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:23.882372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86790c58-af95-4d05-9d49-f72372e90776 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:23.885022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:23.886505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:23.894909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:23.921435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:23.922196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:23.922480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:23.922760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:23.933418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:23.939106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:23.939939 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 58/108] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:42:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:23.941066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:23.998400 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 51/109] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:23.999168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.056888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb2352a-a032-4de3-a30f-9bd1d7fca725 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.058093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:24.094079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.095043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.095043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.095305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/ec0fa36db19a4b8ba2755d8066da3c95/roles/36a32b37790246199c98277b5c065d58` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.097262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] RBAC: Authorizing `identity:create_grant(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76, user_id=ec0fa36db19a4b8ba2755d8066da3c95, role_id=36a32b37790246199c98277b5c065d58)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.098731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.108146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ead64-59d1-49a6-afbf-5e9a6c3a10e9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.120335 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 59/110] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:23 2022] PUT /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/users/ec0fa36db19a4b8ba2755d8066da3c95/roles/36a32b37790246199c98277b5c065d58 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:24.120335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.146676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.147326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.147602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.147877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.160939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:24.165687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:24.166126 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:24.166957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:24.167812 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 52/111] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:42:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:24.168568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.278523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af538bbe-2fe5-444c-89c2-e910b65bbd25 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.288192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21907313-50d7-440f-ba0f-cbb9f54a354a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.304218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:24.333186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.334011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.334339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.334719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.337358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.337680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.338216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.338216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.339026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:24.339193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.340573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.347955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:24.353444 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 53/112] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:24.353809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:24.354615 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:24.355430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:24.355987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.356052 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 60/113] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:24.356052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.465545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1138e4-4d41-497b-88f5-6345a3f982b3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.476153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:24.486999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89ce381-8201-440e-a100-f0c44307b358 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.507346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.508818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.509266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.509685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.511736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.513358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.515157 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 54/114] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:24.516358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.526488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.527111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.527398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.527663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.529046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:24.529303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.530715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.542546 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 61/115] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:24.547858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.572613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:24.654867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d57a5-bb6f-46fd-9dd1-1628676c2fb0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.667212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e1d9f4-73c4-48e2-82a1-a240f3942ec2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.701515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.702150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.702429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.702700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] PATH_INFO: `/v3/users/c668a27e9bfc403386287995d4c4bda0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.703332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.703825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] RBAC: Authorizing `identity:get_user(user_id=c668a27e9bfc403386287995d4c4bda0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.704387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.704896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.705299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.705563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.706419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:24.706915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.708542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.712216 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 62/116] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/users/c668a27e9bfc403386287995d4c4bda0 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:24.713064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.720141 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 55/117] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:24.721257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.736666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:24.822516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff40f1-1bde-4869-aca1-07c3e127e603 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.865637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10010623-9cd7-45ca-8f3d-db758070ed58 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:24.873813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:24.878696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.879849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.880355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.881064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.882766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] RBAC: Adding query filter params (role.id=f6e93371f84c47b0b0ecf09d4467080d, scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76, user.id=0b5a204a6bc6400198969e6cf752082a) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:24.884024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.886337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.894722 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 63/118] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/role_assignments?user.id=0b5a204a6bc6400198969e6cf752082a&scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76&role.id=f6e93371f84c47b0b0ecf09d4467080d => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:24.896131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:24.933125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:24.933783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:24.934136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:24.934413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] PATH_INFO: `/v3/users/0b5a204a6bc6400198969e6cf752082a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:24.935636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] RBAC: Authorizing `identity:get_user(user_id=0b5a204a6bc6400198969e6cf752082a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:24.937111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:24.939065 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 56/119] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/users/0b5a204a6bc6400198969e6cf752082a => generated 275 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:24.939660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:25.043720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:25.072800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e157e2c-7e75-4f14-8afc-7d1198ba2b0d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:25.117496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:25.119779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.121165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.121642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.161780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf90b2b-2b83-4e9a-ac00-7f0b3d3a6d78 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:25.201309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:25.202534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:25.202757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:25.203574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.204050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.204786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] PATH_INFO: `/v3/users/0b5a204a6bc6400198969e6cf752082a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.207067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] RBAC: Authorizing `identity:get_user(user_id=0b5a204a6bc6400198969e6cf752082a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:25.208757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:25.211081 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 57/120] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/users/0b5a204a6bc6400198969e6cf752082a => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:25.212094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:25.273665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:25.301254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:25.327209 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 64/121] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:25.328226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:25.346432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18523593-ec55-4971-ab8c-69e8b546efa5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:25.388838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:25.390091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.391015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.391646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.443456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aebeef-7569-4642-840d-7134117f3d22 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:25.486126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790743e8-25da-49ec-a893-447f021d1ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:25.486932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790743e8-25da-49ec-a893-447f021d1ebe None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.487342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790743e8-25da-49ec-a893-447f021d1ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.487731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790743e8-25da-49ec-a893-447f021d1ebe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.494244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:25.495885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:25.543840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:25.558718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:25.564656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790743e8-25da-49ec-a893-447f021d1ebe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:25.566236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790743e8-25da-49ec-a893-447f021d1ebe None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:25.623627 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 58/122] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:25.623627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:25.678264 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 65/123] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:25.678264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795031-4774-48c5-9cf5-ed9fa03c6906 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.678264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795031-4774-48c5-9cf5-ed9fa03c6906 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.678264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a795031-4774-48c5-9cf5-ed9fa03c6906 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.678264 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 66/124] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:25.678264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a795031-4774-48c5-9cf5-ed9fa03c6906 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:25.728281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61dfcb2-ebdf-4af9-98aa-c08a69d4bd19 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:25.764151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:25.764487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:25.764580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.764580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.764852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.765963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:25.767409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:25.769038 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 59/125] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:25 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:25.769833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:25.782339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:25.787038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a795031-4774-48c5-9cf5-ed9fa03c6906 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:25.828439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb2b1-6cc0-4f2a-b7c8-2efeca106ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:25.829313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb2b1-6cc0-4f2a-b7c8-2efeca106ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.829608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb2b1-6cc0-4f2a-b7c8-2efeca106ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.829885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb2b1-6cc0-4f2a-b7c8-2efeca106ff0 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.830999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb2b1-6cc0-4f2a-b7c8-2efeca106ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:25.832469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb2b1-6cc0-4f2a-b7c8-2efeca106ff0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:25.834091 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 67/126] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:25 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:25.834091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2110d-36bf-452a-bb56-a5cf58c7f789 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.834091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2110d-36bf-452a-bb56-a5cf58c7f789 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.834091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2110d-36bf-452a-bb56-a5cf58c7f789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.855382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c3e2110d-36bf-452a-bb56-a5cf58c7f789 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:25.856586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3e2110d-36bf-452a-bb56-a5cf58c7f789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:25.883984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa122566-4737-48be-b877-a60238fb3b76 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:25.922612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:25.923242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:25.923520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:25.923791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] PATH_INFO: `/v3/projects/2e28229c023a440a886354a80918e350` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:25.924904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] RBAC: Authorizing `identity:get_project(project_id=2e28229c023a440a886354a80918e350)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:25.925472 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 68/127] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:25.926094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2110d-36bf-452a-bb56-a5cf58c7f789 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:25.937513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:25.943859 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 60/128] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:25 2022] GET /identity/v3/projects/2e28229c023a440a886354a80918e350 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:25.943859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:26.021240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:26.028823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:26.042206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2110d-36bf-452a-bb56-a5cf58c7f789 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:26.053407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a676ec-7848-4d34-be07-506846b09023 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:26.077664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:26.078967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.079437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.079997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.080959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76, user.id=c668a27e9bfc403386287995d4c4bda0) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:26.081457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:26.083162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:26.090286 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 69/129] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:25 2022] GET /identity/v3/role_assignments?user.id=c668a27e9bfc403386287995d4c4bda0&scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:26.090708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:26.091326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.091612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.091884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.092402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:26.092705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] RBAC: Adding query filter params (role.id=633fd4ad499a4f62a5a2cbac0eb861ce, scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76, user.id=0b5a204a6bc6400198969e6cf752082a) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:26.093016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:26.094504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:26.104359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 61/130] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:25 2022] GET /identity/v3/role_assignments?user.id=0b5a204a6bc6400198969e6cf752082a&scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76&role.id=633fd4ad499a4f62a5a2cbac0eb861ce => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:26.213312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f89db89-7891-4c33-b368-d4b13fef45aa None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:26.253194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386365b8-bfdb-40ad-9241-4b737b313120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:26.254207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386365b8-bfdb-40ad-9241-4b737b313120 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.254685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386365b8-bfdb-40ad-9241-4b737b313120 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.255151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386365b8-bfdb-40ad-9241-4b737b313120 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.256495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386365b8-bfdb-40ad-9241-4b737b313120 None admin] RBAC: Adding query filter params (role.id=f6e93371f84c47b0b0ecf09d4467080d, scope.project.id=2e28229c023a440a886354a80918e350, user.id=0b5a204a6bc6400198969e6cf752082a) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:26.257056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386365b8-bfdb-40ad-9241-4b737b313120 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:26.258921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386365b8-bfdb-40ad-9241-4b737b313120 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:26.265970 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 70/131] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:26 2022] GET /identity/v3/role_assignments?user.id=0b5a204a6bc6400198969e6cf752082a&scope.project.id=2e28229c023a440a886354a80918e350&role.id=f6e93371f84c47b0b0ecf09d4467080d => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:26.300800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:26.435619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989949c1-5dc3-432c-bd8c-a8e71911b1a6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:26.445392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e03e74-04d2-48ee-beaf-727d45b1368f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.446539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e03e74-04d2-48ee-beaf-727d45b1368f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.447000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e03e74-04d2-48ee-beaf-727d45b1368f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.448427 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 71/132] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:26.457593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1e5f-20cc-46ea-a9b0-a9f161e1acb7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.457907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1e5f-20cc-46ea-a9b0-a9f161e1acb7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.458180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ec1e5f-20cc-46ea-a9b0-a9f161e1acb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.478405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:26.478996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.479278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.479553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.480756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:26.482239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:26.483671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-43ec1e5f-20cc-46ea-a9b0-a9f161e1acb7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:26.484009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 62/133] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:26.484998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-43ec1e5f-20cc-46ea-a9b0-a9f161e1acb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:26.488558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:26.489901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:26.503198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:26.562972 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 72/134] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:26.598616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d3fc5-c1cd-4d28-bf15-57beda3d65a7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:26.632211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:26.632846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.633156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.633433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.634090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:26.635519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:26.654463 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 63/135] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:42:26 2022] POST /identity/v3/users => generated 283 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:42:26.828951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec1e5f-20cc-46ea-a9b0-a9f161e1acb7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:26.929378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ec1e5f-20cc-46ea-a9b0-a9f161e1acb7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:26.963659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:26.964310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:26.964605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:26.965510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:26.973062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:26.975820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:26.981528 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:26.982389 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:26.983174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:26.987467 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 73/136] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:26.988899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:27.059158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:27.098812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e838a16-f6fa-436f-85d9-8750d7214b57 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:27.139801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:27.140446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:27.140725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:27.140993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:27.141657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:27.141958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:27.143337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:27.152516 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 64/137] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:27.158397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:27.263528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda730d4-9a14-4d6e-90c3-f6390488d173 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:27.299279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:27.299910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:27.300187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:27.300561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] PATH_INFO: `/v3/users/ec0fa36db19a4b8ba2755d8066da3c95` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:27.301747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] RBAC: Authorizing `identity:get_user(user_id=ec0fa36db19a4b8ba2755d8066da3c95)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:27.303183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:27.304775 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 74/138] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:27 2022] GET /identity/v3/users/ec0fa36db19a4b8ba2755d8066da3c95 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:27.311136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:27.422151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec826d7-319f-4bbf-869c-04d6db053a5e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:27.449874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:27.459197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:27.459816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:27.460095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:27.460400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:27.538726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:27.540583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:27.644089 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 65/139] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:27.653089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:27.725161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:27.761381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fea3ed5-4f0f-4812-bb93-4d1b844600d7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:27.795810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff1035-6638-4b87-91af-5b7fa4117fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:27.796483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff1035-6638-4b87-91af-5b7fa4117fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:27.796767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff1035-6638-4b87-91af-5b7fa4117fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:27.797041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff1035-6638-4b87-91af-5b7fa4117fe8 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:27.798171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff1035-6638-4b87-91af-5b7fa4117fe8 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:27.799632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff1035-6638-4b87-91af-5b7fa4117fe8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:27.801290 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 75/140] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:27 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:27.807935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:27.843017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:27.919638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604164c-29e0-48bd-b90f-1b094221fdb6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:27.957602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0214ff96-15dd-4bc7-a363-ee00e5feed31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:27.958578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ff96-15dd-4bc7-a363-ee00e5feed31 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:27.959065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ff96-15dd-4bc7-a363-ee00e5feed31 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:27.959554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0214ff96-15dd-4bc7-a363-ee00e5feed31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:27.960521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214ff96-15dd-4bc7-a363-ee00e5feed31 None admin] RBAC: Adding query filter params (role.id=36a32b37790246199c98277b5c065d58, scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76, user.id=ec0fa36db19a4b8ba2755d8066da3c95) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:27.961065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214ff96-15dd-4bc7-a363-ee00e5feed31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:27.962700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0214ff96-15dd-4bc7-a363-ee00e5feed31 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:27.969011 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 66/141] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:27 2022] GET /identity/v3/role_assignments?user.id=ec0fa36db19a4b8ba2755d8066da3c95&scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76&role.id=36a32b37790246199c98277b5c065d58 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:28.966369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1757ad6-faf2-4737-a3fc-22e82b5536b4 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:28.966896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1757ad6-faf2-4737-a3fc-22e82b5536b4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:28.966896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1757ad6-faf2-4737-a3fc-22e82b5536b4 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:28.968229 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 76/142] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:28.973528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460e35-586f-416c-844d-80eb2ff07d61 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:28.973916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460e35-586f-416c-844d-80eb2ff07d61 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:28.974089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7460e35-586f-416c-844d-80eb2ff07d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:28.992124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b7460e35-586f-416c-844d-80eb2ff07d61 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:28.993436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b7460e35-586f-416c-844d-80eb2ff07d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:29.064767 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 67/143] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:29.141700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46326283-aa38-4541-aa7c-ef8b5fa7fa9d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.141887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46326283-aa38-4541-aa7c-ef8b5fa7fa9d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.142152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46326283-aa38-4541-aa7c-ef8b5fa7fa9d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.143621 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 77/144] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:29.149034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e674ca-f3a7-4ff6-96ff-c464d46a40d7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.149327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e674ca-f3a7-4ff6-96ff-c464d46a40d7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.149598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e674ca-f3a7-4ff6-96ff-c464d46a40d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.167392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f7e674ca-f3a7-4ff6-96ff-c464d46a40d7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:29.168576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7e674ca-f3a7-4ff6-96ff-c464d46a40d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:29.185493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:29.205409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f3e0-9c28-44f9-b377-b84124054c62 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.205686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f3e0-9c28-44f9-b377-b84124054c62 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.205970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c407f3e0-9c28-44f9-b377-b84124054c62 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.207291 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 78/145] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:29.212584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bbf9e-97db-41b9-8ff2-dd19f39bc444 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.212866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bbf9e-97db-41b9-8ff2-dd19f39bc444 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.213139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bbf9e-97db-41b9-8ff2-dd19f39bc444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.231752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9a3bbf9e-97db-41b9-8ff2-dd19f39bc444 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:29.233063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9a3bbf9e-97db-41b9-8ff2-dd19f39bc444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:29.240025 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 68/146] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:29.300067 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 79/147] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:29.331638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e674ca-f3a7-4ff6-96ff-c464d46a40d7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.426653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e674ca-f3a7-4ff6-96ff-c464d46a40d7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:29.460323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:29.460961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.461854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.462122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.472320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:29.477074 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:29.477866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:29.478661 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:29.480605 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 69/148] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:29.483928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bbf9e-97db-41b9-8ff2-dd19f39bc444 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.518614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.534510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:29.556460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:29.592525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3bbf9e-97db-41b9-8ff2-dd19f39bc444 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:29.618604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54200a66-3f03-4667-9162-f90b9189750f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:29.624761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:29.625364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.625650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.625926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.626582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:29.626884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:29.628245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:29.636896 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 80/149] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:29.636896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.653567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:29.654202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.654486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.654762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.665953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:29.670768 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:29.671705 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 70/150] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:29.672808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.738562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a78bb-1ce3-49fb-931a-b41b89948abb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:29.740160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:29.764481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:29.771761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e1a742-335f-4298-a924-9a9b4aa3a67c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:29.772677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:29.773304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.773609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.773882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.774994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:29.776451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:29.778037 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 81/151] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:29.779886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.806764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:29.807392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.807670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.807940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] PATH_INFO: `/v3/users/a83928bc02b64987a984cf98954e9ab5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.821465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] RBAC: Authorizing `identity:get_user(user_id=a83928bc02b64987a984cf98954e9ab5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:29.822939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:29.824600 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 71/152] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/users/a83928bc02b64987a984cf98954e9ab5 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:29.829249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.890940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e8656-6696-40cf-ad18-c9c69af21a10 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:29.923631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:29.923799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad088d-6b61-4b34-afdb-ecce4d1d4975 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:29.924236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.924544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.924826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.925488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:29.926432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:29.927177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:29.935379 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 82/153] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:29.937388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:29.952635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:29.958702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:29.959313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:29.959592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:29.959864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:29.960537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:29.961957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:29.982942 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 72/154] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:42:29 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:29.984437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:30.004593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:30.056453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb23fbf2-3c23-4184-afd6-55794fc86d90 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:30.094517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:30.095222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:30.095614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:30.095921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:30.097224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:30.098702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:30.099280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dd5e6f-0c9f-4b3c-9302-b94516629b42 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:30.100595 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 83/155] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:30.106745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:30.138397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:30.139052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:30.139323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:30.139598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] PATH_INFO: `/v3/users/a83928bc02b64987a984cf98954e9ab5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:30.140813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] RBAC: Authorizing `identity:get_user(user_id=a83928bc02b64987a984cf98954e9ab5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:30.142281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:30.143925 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 73/156] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:29 2022] GET /identity/v3/users/a83928bc02b64987a984cf98954e9ab5 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:30.149662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:30.213179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:30.233804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfa966-d64f-46cf-9868-9305e2aded83 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:30.266010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3366a-f50a-4711-838c-3bdd2a234961 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:30.267487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:30.279854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:30.280822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:30.280822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:30.281133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891/users/a83928bc02b64987a984cf98954e9ab5/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:30.284584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] RBAC: Authorizing `identity:create_grant(project_id=9abcb12b59954bc4b95706027f575891, user_id=a83928bc02b64987a984cf98954e9ab5, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:30.286050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:30.304073 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 84/157] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:30 2022] PUT /identity/v3/projects/9abcb12b59954bc4b95706027f575891/users/a83928bc02b64987a984cf98954e9ab5/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:42:30.312093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297e9152-24e0-4a92-9614-8e17f4188506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:30.312919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9152-24e0-4a92-9614-8e17f4188506 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:30.313262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9152-24e0-4a92-9614-8e17f4188506 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:30.313603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297e9152-24e0-4a92-9614-8e17f4188506 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:30.314909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e9152-24e0-4a92-9614-8e17f4188506 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:30.316891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297e9152-24e0-4a92-9614-8e17f4188506 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:30.318520 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 74/158] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:30 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:30.324791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:30.448409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce5af52-44a7-42f7-9931-5d6a65c5de54 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:30.489069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6040a1a-0917-49f6-96fd-5a9d0a7506a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:30.489746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6040a1a-0917-49f6-96fd-5a9d0a7506a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:30.490081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6040a1a-0917-49f6-96fd-5a9d0a7506a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:30.490366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6040a1a-0917-49f6-96fd-5a9d0a7506a2 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891/users/a83928bc02b64987a984cf98954e9ab5/roles/633fd4ad499a4f62a5a2cbac0eb861ce` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:30.492605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6040a1a-0917-49f6-96fd-5a9d0a7506a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=9abcb12b59954bc4b95706027f575891, user_id=a83928bc02b64987a984cf98954e9ab5, role_id=633fd4ad499a4f62a5a2cbac0eb861ce)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:30.494056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6040a1a-0917-49f6-96fd-5a9d0a7506a2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:30.510066 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 85/159] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:30 2022] PUT /identity/v3/projects/9abcb12b59954bc4b95706027f575891/users/a83928bc02b64987a984cf98954e9ab5/roles/633fd4ad499a4f62a5a2cbac0eb861ce => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:32.260187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119caf3-5608-4d6b-b951-a16e608ecbee None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.260731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119caf3-5608-4d6b-b951-a16e608ecbee None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.260731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9119caf3-5608-4d6b-b951-a16e608ecbee None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.262353 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 75/160] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:32.268151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9276dd-ebe5-4606-b20e-915abd0929d7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.268689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9276dd-ebe5-4606-b20e-915abd0929d7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.268784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9276dd-ebe5-4606-b20e-915abd0929d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.288439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8d9276dd-ebe5-4606-b20e-915abd0929d7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:32.290344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8d9276dd-ebe5-4606-b20e-915abd0929d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:32.335450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:32.362494 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 86/161] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:32.406573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0adc2-41a5-4e12-ac78-c418e29facb4 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.406852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0adc2-41a5-4e12-ac78-c418e29facb4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.407113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0adc2-41a5-4e12-ac78-c418e29facb4 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.408618 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 76/162] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:32.413688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b0d8-4009-484f-999d-76d349c222d1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.413991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b0d8-4009-484f-999d-76d349c222d1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.414268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6436b0d8-4009-484f-999d-76d349c222d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.433526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6436b0d8-4009-484f-999d-76d349c222d1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:32.434735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6436b0d8-4009-484f-999d-76d349c222d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:32.504806 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 87/163] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:32.563632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917bf30-9b00-4321-a638-0cab30500c89 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.563936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917bf30-9b00-4321-a638-0cab30500c89 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.564187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917bf30-9b00-4321-a638-0cab30500c89 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.565818 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 77/164] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:32.571555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2910f3-d254-41cb-b9cb-fa41bd1d13da None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.571839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2910f3-d254-41cb-b9cb-fa41bd1d13da None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.572113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2910f3-d254-41cb-b9cb-fa41bd1d13da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.591041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7a2910f3-d254-41cb-b9cb-fa41bd1d13da None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:32.592333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7a2910f3-d254-41cb-b9cb-fa41bd1d13da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:32.656813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917bf30-9b00-4321-a638-0cab30500c89 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:32.664784 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 88/165] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:32.722025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a55ec-c22a-4575-a720-ce31a8cee304 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.722401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a55ec-c22a-4575-a720-ce31a8cee304 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.722684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357a55ec-c22a-4575-a720-ce31a8cee304 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.724153 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 89/166] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:42:32.729396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf6aff-44aa-4053-9b91-feb39b924a23 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.729850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf6aff-44aa-4053-9b91-feb39b924a23 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.730366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bf6aff-44aa-4053-9b91-feb39b924a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.749657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a9bf6aff-44aa-4053-9b91-feb39b924a23 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:32.751181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a9bf6aff-44aa-4053-9b91-feb39b924a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:32.767154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917bf30-9b00-4321-a638-0cab30500c89 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:32.787959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:32.790595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:32.804469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:32.805734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.805946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.806207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.818436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:32.823868 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:32.824937 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:32.825879 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:32.829959 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 78/167] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:32.830681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:32.835873 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 90/168] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:32.836965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf6aff-44aa-4053-9b91-feb39b924a23 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:32.951292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5beac55f-2e24-44dd-a36a-308cf2bf117e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:32.955276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bf6aff-44aa-4053-9b91-feb39b924a23 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:32.988394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:32.988566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:32.989001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.989129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:32.989280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.989368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:32.989538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.989626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:32.990188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:32.990485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:32.990663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:32.991833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:32.992658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:32.996355 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 79/169] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:32.996355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.007601 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 91/170] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:33.007601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.033116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.052850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.107221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a460575f-c12e-44e5-a931-a0d5e463df88 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.109524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136710da-6964-45af-82b1-30bc3d87c30a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.140824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.141457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.141743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.142012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.143175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.143777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.144055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.144349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.144996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.146423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:33.151975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:33.157822 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:33.158678 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:33.159470 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 80/171] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:42:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:33.160134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.180335 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 92/172] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:42:33 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:33.180335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.269533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.286357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f5088c-364f-4fae-b6e1-3a86d10a62ce None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.289803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274025cc-a2c6-4826-9f69-f126f867e8fb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.299935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.328110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.328792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.329063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.329337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] PATH_INFO: `/v3/users/75e4c4f0b70d47cc831e2441b1166c9e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.331006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.331943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.332501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.332829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.344005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] RBAC: Authorizing `identity:get_user(user_id=75e4c4f0b70d47cc831e2441b1166c9e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.345503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:33.345632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:33.348336 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 81/173] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/users/75e4c4f0b70d47cc831e2441b1166c9e => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:33.348336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:33.352090 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:33.353145 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:33.354022 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 93/174] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:33.354647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.466935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6b778-146b-4e1e-9cff-cfd9f41fb866 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.484914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c0f81-4c66-411d-a5f7-a9124570be1a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.511737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.512532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.512911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.513872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.514122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:33.514465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.515925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:33.524859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.525621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.526000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.526370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.526657 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 82/175] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:33.527485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.527854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.529411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:33.531699 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 94/176] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:33.534399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.551631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.566033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.647031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa27504-9027-4b02-bdfc-40f384eddbd8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.660267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22bf5-dbfa-4a67-bcf1-0168d893dd10 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.684885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.685985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.686500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.686988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] PATH_INFO: `/v3/users/a83928bc02b64987a984cf98954e9ab5` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.688761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] RBAC: Authorizing `identity:get_user(user_id=a83928bc02b64987a984cf98954e9ab5)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.690222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:33.691828 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 95/177] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/users/a83928bc02b64987a984cf98954e9ab5 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:33.692611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.706723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.707343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.707623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.707893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.708589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:33.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.710280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:33.719995 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 83/178] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:33.720794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.785406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.808681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77521fea-1084-4b5e-8df7-e3a238af6402 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.833277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:33.849729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568d533b-86c6-478e-b7aa-4953c869d926 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:33.854384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.855222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.855684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.856107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891/users/75e4c4f0b70d47cc831e2441b1166c9e/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.858242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] RBAC: Authorizing `identity:create_grant(project_id=9abcb12b59954bc4b95706027f575891, user_id=75e4c4f0b70d47cc831e2441b1166c9e, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.859891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:33.876217 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 96/179] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:33 2022] PUT /identity/v3/projects/9abcb12b59954bc4b95706027f575891/users/75e4c4f0b70d47cc831e2441b1166c9e/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:33.878242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:33.891733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:33.892415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:33.892726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:33.893040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:33.989398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:33.990813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:34.045584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384ee8e-9c3b-4e71-8cf1-aae24bb50781 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:34.107018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:34.107765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:34.108129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:34.108482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] PATH_INFO: `/v3/users/a83928bc02b64987a984cf98954e9ab5` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:34.110762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] RBAC: Authorizing `identity:get_user(user_id=a83928bc02b64987a984cf98954e9ab5)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:34.112264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:34.114319 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 97/180] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/users/a83928bc02b64987a984cf98954e9ab5 => generated 283 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:34.119854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:34.136573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:34.170560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:34.187229 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 84/181] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:34.193712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:34.259721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154fce9-382c-44ae-b123-9b790586a8fd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:34.327928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ade43-331b-40ca-83b6-624ac91719aa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:34.336898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:34.338049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:34.344322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:34.344861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:34.365656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:34.366435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:34.366721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:34.366995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:34.369078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:34.370501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:34.372342 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 85/182] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:34 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:34.378128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:34.444798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:34.490412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba4c1a4-8451-4d5a-a02d-2e68c3947112 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:34.501484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:34.502983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:34.520844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:34.528924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:34.529542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:34.529822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:34.530094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:34.530819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=9abcb12b59954bc4b95706027f575891, user.id=a83928bc02b64987a984cf98954e9ab5) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:34.531130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:34.532584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:34.539256 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 86/183] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:34 2022] GET /identity/v3/role_assignments?user.id=a83928bc02b64987a984cf98954e9ab5&scope.project.id=9abcb12b59954bc4b95706027f575891&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:34.639445 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 98/184] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:34.644965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:34.763153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10725118-4fb4-4c53-9417-ccf1b7f058f5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:34.792802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:34.818386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecdd0fd-2865-4631-8319-2a81f5f322a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:34.819133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdd0fd-2865-4631-8319-2a81f5f322a2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:34.819483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdd0fd-2865-4631-8319-2a81f5f322a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:34.819826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecdd0fd-2865-4631-8319-2a81f5f322a2 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:34.821221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdd0fd-2865-4631-8319-2a81f5f322a2 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:34.822799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecdd0fd-2865-4631-8319-2a81f5f322a2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:34.824573 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 87/185] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:34 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:34.830659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:34.942693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe903b-aa62-413a-96b8-22def4fc1d4d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:34.952680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:34.982045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c8228-cae1-4dc0-8ee2-cf0cc839611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:34.982724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c8228-cae1-4dc0-8ee2-cf0cc839611a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:34.983010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c8228-cae1-4dc0-8ee2-cf0cc839611a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:34.983285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c8228-cae1-4dc0-8ee2-cf0cc839611a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:34.984040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c8228-cae1-4dc0-8ee2-cf0cc839611a None admin] RBAC: Adding query filter params (role.id=633fd4ad499a4f62a5a2cbac0eb861ce, scope.project.id=9abcb12b59954bc4b95706027f575891, user.id=a83928bc02b64987a984cf98954e9ab5) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:34.984377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c8228-cae1-4dc0-8ee2-cf0cc839611a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:34.985902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c8228-cae1-4dc0-8ee2-cf0cc839611a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:34.995107 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 99/186] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:34 2022] GET /identity/v3/role_assignments?user.id=a83928bc02b64987a984cf98954e9ab5&scope.project.id=9abcb12b59954bc4b95706027f575891&role.id=633fd4ad499a4f62a5a2cbac0eb861ce => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:35.678927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797f63b-22e0-4035-9f6d-29a9eaea9a2f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:35.679444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797f63b-22e0-4035-9f6d-29a9eaea9a2f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:35.679444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797f63b-22e0-4035-9f6d-29a9eaea9a2f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:35.681838 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 88/187] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:35.686842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52730e83-0e81-478f-99f1-b425bb2aa83d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:35.687258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52730e83-0e81-478f-99f1-b425bb2aa83d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:35.687449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52730e83-0e81-478f-99f1-b425bb2aa83d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:35.705873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-52730e83-0e81-478f-99f1-b425bb2aa83d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:35.707032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-52730e83-0e81-478f-99f1-b425bb2aa83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:35.778081 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 100/188] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:35.869445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf62fb-0e72-4adf-940a-b938735cd522 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:35.869763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf62fb-0e72-4adf-940a-b938735cd522 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:35.869989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf62fb-0e72-4adf-940a-b938735cd522 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:35.871468 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 89/189] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:35.876512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dd330-d74e-4a9d-ac22-2f260d9eb4bf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:35.876798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dd330-d74e-4a9d-ac22-2f260d9eb4bf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:35.877069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8dd330-d74e-4a9d-ac22-2f260d9eb4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:35.895741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4e8dd330-d74e-4a9d-ac22-2f260d9eb4bf None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:35.896942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4e8dd330-d74e-4a9d-ac22-2f260d9eb4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:35.961673 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 101/190] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:36.045150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf62fb-0e72-4adf-940a-b938735cd522 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.143822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf62fb-0e72-4adf-940a-b938735cd522 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.179131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.180449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.180730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.181001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.193554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:36.198502 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:36.199347 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:36.200135 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 90/191] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:36.206767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dd330-d74e-4a9d-ac22-2f260d9eb4bf None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.238491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:36.287993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:36.308203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8dd330-d74e-4a9d-ac22-2f260d9eb4bf None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.334073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df41d4-b46a-465c-b34c-fdf92088ac00 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.341083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec41b0e-61c6-4ae1-b051-025cfbcdc835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.341701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41b0e-61c6-4ae1-b051-025cfbcdc835 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.341980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41b0e-61c6-4ae1-b051-025cfbcdc835 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.342253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec41b0e-61c6-4ae1-b051-025cfbcdc835 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.342921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec41b0e-61c6-4ae1-b051-025cfbcdc835 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:36.343227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec41b0e-61c6-4ae1-b051-025cfbcdc835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:36.344651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec41b0e-61c6-4ae1-b051-025cfbcdc835 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:36.355027 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 102/192] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:36.356176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9842-6806-435a-b8d3-24ef90231b91 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.356497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9842-6806-435a-b8d3-24ef90231b91 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.356770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9842-6806-435a-b8d3-24ef90231b91 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.357999 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 103/193] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:36.360941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9842-6806-435a-b8d3-24ef90231b91 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.372762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19a2ad-58d7-4600-90f0-45158df4d4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.373391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a2ad-58d7-4600-90f0-45158df4d4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.373666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a2ad-58d7-4600-90f0-45158df4d4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.373932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19a2ad-58d7-4600-90f0-45158df4d4f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.375017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19a2ad-58d7-4600-90f0-45158df4d4f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:36.376485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19a2ad-58d7-4600-90f0-45158df4d4f7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:36.378035 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 91/194] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:36.383988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcde2b-8d8a-4ad8-9b35-f4f339ebf815 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.384272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcde2b-8d8a-4ad8-9b35-f4f339ebf815 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.384564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafcde2b-8d8a-4ad8-9b35-f4f339ebf815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.402726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-eafcde2b-8d8a-4ad8-9b35-f4f339ebf815 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:36.403913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eafcde2b-8d8a-4ad8-9b35-f4f339ebf815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:36.454551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:36.463098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9842-6806-435a-b8d3-24ef90231b91 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.472739 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 92/195] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:36.473570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcde2b-8d8a-4ad8-9b35-f4f339ebf815 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.498150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.498854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.499129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.499400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] PATH_INFO: `/v3/users/748d982757604c0389b1110d28e0f5ed` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.513021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] RBAC: Authorizing `identity:get_user(user_id=748d982757604c0389b1110d28e0f5ed)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:36.514456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:36.516242 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 104/196] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:36 2022] GET /identity/v3/users/748d982757604c0389b1110d28e0f5ed => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:36.521398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.528259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:36.581829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafcde2b-8d8a-4ad8-9b35-f4f339ebf815 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.621555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.622331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.622646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.622894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.623543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:36.625009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:36.626514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec91daa-adcf-43ef-80cb-de116bc368ae None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.656332 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 93/197] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:42:36 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:36.673308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.674095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.674531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.674872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.676224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:36.677850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:36.680494 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 105/198] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:36 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:36.686201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.746780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:36.777017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:36.805614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c3248-9e07-4aa8-8a03-49f9417402fe None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.815993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:36.853058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.853723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.854005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.854275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891/users/748d982757604c0389b1110d28e0f5ed/roles/36a32b37790246199c98277b5c065d58` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.856193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] RBAC: Authorizing `identity:create_grant(project_id=9abcb12b59954bc4b95706027f575891, user_id=748d982757604c0389b1110d28e0f5ed, role_id=36a32b37790246199c98277b5c065d58)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:36.859164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:36.873912 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 94/199] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:42:36 2022] PUT /identity/v3/projects/9abcb12b59954bc4b95706027f575891/users/748d982757604c0389b1110d28e0f5ed/roles/36a32b37790246199c98277b5c065d58 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:36.879237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5f494-e445-40c6-b253-9a4d837e6f54 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:36.924218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:36.924927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:36.925182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:36.925452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:36.939105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:36.945815 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:36.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:36.947924 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:36.949726 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 106/200] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:36.955407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:37.076420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e259bc6-5c37-44ad-aedd-4fc22cbccc87 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:37.110607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:37.111217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:37.111490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:37.111755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:37.112440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:37.112793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:37.114268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:37.122927 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 95/201] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:37.128508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:37.215189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:37.236346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17e5c0-9704-4f39-a66e-5b2842a035f1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:37.274788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:37.275768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:37.276274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:37.276769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] PATH_INFO: `/v3/users/75e4c4f0b70d47cc831e2441b1166c9e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:37.278248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] RBAC: Authorizing `identity:get_user(user_id=75e4c4f0b70d47cc831e2441b1166c9e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:37.279969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:37.282149 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 107/202] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:37 2022] GET /identity/v3/users/75e4c4f0b70d47cc831e2441b1166c9e => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:37.288411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:37.321472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:37.397631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddcea6-582e-413f-a43f-850ff524ba82 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:37.433124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:37.433968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:37.434393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:37.434838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:37.504246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:37.505985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:37.534179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:37.609247 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 96/203] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:37.618314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:37.720911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f600594-b77a-47c3-921d-d9a0cc75bede None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:37.755624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2cd49-7ae2-4b5f-a845-9c85cb87fee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:37.756513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2cd49-7ae2-4b5f-a845-9c85cb87fee7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:37.756800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2cd49-7ae2-4b5f-a845-9c85cb87fee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:37.757067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2cd49-7ae2-4b5f-a845-9c85cb87fee7 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:37.758292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2cd49-7ae2-4b5f-a845-9c85cb87fee7 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:37.759772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2cd49-7ae2-4b5f-a845-9c85cb87fee7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:37.761703 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 108/204] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:37 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:37.767443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:37.878481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c62228-0c09-4730-8cb3-5c1472ed3247 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:37.908605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:37.917771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb1fca-7df5-4337-8920-ea01a0610f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:37.918625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb1fca-7df5-4337-8920-ea01a0610f5c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:37.919107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb1fca-7df5-4337-8920-ea01a0610f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:37.919598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb1fca-7df5-4337-8920-ea01a0610f5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:37.920603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb1fca-7df5-4337-8920-ea01a0610f5c None admin] RBAC: Adding query filter params (role.id=f6e93371f84c47b0b0ecf09d4467080d, scope.project.id=9abcb12b59954bc4b95706027f575891, user.id=75e4c4f0b70d47cc831e2441b1166c9e) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:37.921121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb1fca-7df5-4337-8920-ea01a0610f5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:37.922738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb1fca-7df5-4337-8920-ea01a0610f5c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:37.929017 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 97/205] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:37 2022] GET /identity/v3/role_assignments?user.id=75e4c4f0b70d47cc831e2441b1166c9e&scope.project.id=9abcb12b59954bc4b95706027f575891&role.id=f6e93371f84c47b0b0ecf09d4467080d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:39.090140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e639ed-fc6b-41c9-90e5-5a1beb5cc79d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.090624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e639ed-fc6b-41c9-90e5-5a1beb5cc79d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.090624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e639ed-fc6b-41c9-90e5-5a1beb5cc79d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.092041 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 109/206] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:39.092643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c70c4-f45c-4a2a-8e73-3354f7f414e1 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.093010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c70c4-f45c-4a2a-8e73-3354f7f414e1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.093199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c70c4-f45c-4a2a-8e73-3354f7f414e1 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.094563 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 98/207] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:42:39.096939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378560a0-eb96-41ce-a2bb-0dd8601dce8a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.097219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378560a0-eb96-41ce-a2bb-0dd8601dce8a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.097490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378560a0-eb96-41ce-a2bb-0dd8601dce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.099380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2932-33a1-44fa-8687-27352715f653 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.099663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2932-33a1-44fa-8687-27352715f653 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.099939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695c2932-33a1-44fa-8687-27352715f653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.115923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-378560a0-eb96-41ce-a2bb-0dd8601dce8a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:39.117680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-378560a0-eb96-41ce-a2bb-0dd8601dce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:39.120097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-695c2932-33a1-44fa-8687-27352715f653 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:39.121286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-695c2932-33a1-44fa-8687-27352715f653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:39.128184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:39.183320 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 110/208] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:39.184636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea010a1-7133-4a97-82bd-e5e8226fa3b5 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.184915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea010a1-7133-4a97-82bd-e5e8226fa3b5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.185185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea010a1-7133-4a97-82bd-e5e8226fa3b5 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.186418 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 111/209] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:39.186970 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 99/210] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:39.193680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea88a5-8a8f-4291-a2fd-f013a885fe85 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.194029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea88a5-8a8f-4291-a2fd-f013a885fe85 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.194367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea88a5-8a8f-4291-a2fd-f013a885fe85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.211562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3aea88a5-8a8f-4291-a2fd-f013a885fe85 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:39.212744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3aea88a5-8a8f-4291-a2fd-f013a885fe85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:39.277230 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 112/211] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:39.456399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c2932-33a1-44fa-8687-27352715f653 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.462255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea88a5-8a8f-4291-a2fd-f013a885fe85 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.544156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:39.559343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695c2932-33a1-44fa-8687-27352715f653 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:39.566874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea88a5-8a8f-4291-a2fd-f013a885fe85 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:39.581243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:39.592856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:39.593480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.593749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.594016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.602032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:39.602654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.602928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.603194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.604238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:39.609012 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:39.609925 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:39.610794 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 100/212] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:39.611433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.615055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:39.620186 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:39.621263 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:39.622048 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 113/213] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:39.622684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.712658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c5d48-53b4-4532-92ef-cca68c518892 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:39.716556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75a98bc-5a31-461b-a5b0-32c33fff8b33 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:39.745912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:39.746530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.746802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.747063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.748131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:39.749016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:39.749590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:39.749679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.749878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.750147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.750801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:39.751102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:39.751220 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 101/214] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:39.752645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:39.754869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:39.756632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.766438 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 114/215] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:39.766438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.804799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:39.859504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795e2a6-aadc-4747-a88e-2de6fc080cdb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:39.862464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60079c-ef19-438f-84f1-ecf4fc50a73f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:39.893458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:39.894062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.894341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.894611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.895055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:39.895266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:39.895565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:39.895652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:39.895922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:39.896191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:39.896858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:39.896990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:39.898859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:39.905647 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 102/216] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:39.905647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.919624 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 115/217] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:42:39 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:42:39.920493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:39.981945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:40.029668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077e3cd0-91c1-4d7c-9bc5-b6268a0d621e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:40.031289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd88410-872b-4078-bf5a-41679a77853b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:40.044108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:40.067931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:40.068638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:40.068920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:40.069193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] PATH_INFO: `/v3/users/cffd6287003746e09944f5d491a9fed5` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:40.069767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:40.070993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:40.071268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:40.071536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] PATH_INFO: `/v3/users/748d982757604c0389b1110d28e0f5ed` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:40.072970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] RBAC: Authorizing `identity:get_user(user_id=748d982757604c0389b1110d28e0f5ed)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:40.074431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:40.080751 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 103/218] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:39 2022] GET /identity/v3/users/748d982757604c0389b1110d28e0f5ed => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:40.081859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:40.088494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] RBAC: Authorizing `identity:get_user(user_id=cffd6287003746e09944f5d491a9fed5)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:40.089961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:40.091893 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 116/219] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:39 2022] GET /identity/v3/users/cffd6287003746e09944f5d491a9fed5 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:40.097369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:40.202330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a95074-af37-4566-8d4a-9fcf4c997042 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:40.222286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47389489-4ba3-4e97-a010-55cced4bfcfc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:40.238260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:40.253979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:40.255040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:40.255530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:40.255808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:40.269074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:40.269796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:40.270093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:40.270364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:40.271100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] RBAC: Adding query filter params (role.id=f6e93371f84c47b0b0ecf09d4467080d, user.id=cffd6287003746e09944f5d491a9fed5, scope.system=all) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:40.271410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:40.274633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:40.280187 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 117/220] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Oct 23 00:42:40 2022] GET /identity/v3/role_assignments?user.id=cffd6287003746e09944f5d491a9fed5&scope.system=all&role.id=f6e93371f84c47b0b0ecf09d4467080d => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:40.318682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:40.342382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:40.343964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:40.463739 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 104/221] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:40.469916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:40.580140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa9074-5ec4-4253-9aa9-7551aeb160a8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:40.614860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc90825-b137-48c9-9103-02b96c17be69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:40.615495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc90825-b137-48c9-9103-02b96c17be69 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:40.615770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc90825-b137-48c9-9103-02b96c17be69 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:40.616038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc90825-b137-48c9-9103-02b96c17be69 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:40.617257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc90825-b137-48c9-9103-02b96c17be69 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:40.618705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc90825-b137-48c9-9103-02b96c17be69 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:40.620457 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 118/222] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:40 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:40.626424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:40.710120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:40.731247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eecf64-4c71-458b-a19e-e521add7fcd8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:40.767640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ffb3f-296c-48cc-bac2-a4dcbfaecc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:40.768330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ffb3f-296c-48cc-bac2-a4dcbfaecc59 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:40.768615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ffb3f-296c-48cc-bac2-a4dcbfaecc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:40.768887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ffb3f-296c-48cc-bac2-a4dcbfaecc59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:40.769612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ffb3f-296c-48cc-bac2-a4dcbfaecc59 None admin] RBAC: Adding query filter params (role.id=36a32b37790246199c98277b5c065d58, scope.project.id=9abcb12b59954bc4b95706027f575891, user.id=748d982757604c0389b1110d28e0f5ed) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:40.769979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ffb3f-296c-48cc-bac2-a4dcbfaecc59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:40.771479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ffb3f-296c-48cc-bac2-a4dcbfaecc59 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:40.778043 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 105/223] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:42:40 2022] GET /identity/v3/role_assignments?user.id=748d982757604c0389b1110d28e0f5ed&scope.project.id=9abcb12b59954bc4b95706027f575891&role.id=36a32b37790246199c98277b5c065d58 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:42.211272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a0e4-5675-421a-9db0-8b1d30313a31 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.211771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a0e4-5675-421a-9db0-8b1d30313a31 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.211771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a0e4-5675-421a-9db0-8b1d30313a31 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.213209 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 119/224] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:42.219013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eebca9-9d70-4a4d-8e7a-e37f5c6a7b17 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.219551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eebca9-9d70-4a4d-8e7a-e37f5c6a7b17 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.219551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eebca9-9d70-4a4d-8e7a-e37f5c6a7b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.238807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-35eebca9-9d70-4a4d-8e7a-e37f5c6a7b17 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:42.240003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-35eebca9-9d70-4a4d-8e7a-e37f5c6a7b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:42.309084 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 106/225] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:42.322874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421a333-28fc-476f-9e8b-b5a7a5db63c2 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.323159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421a333-28fc-476f-9e8b-b5a7a5db63c2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.323433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421a333-28fc-476f-9e8b-b5a7a5db63c2 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.324813 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 120/226] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:42.330066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8356a-0df8-407a-ade3-10ac87e02de7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.330354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8356a-0df8-407a-ade3-10ac87e02de7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.330628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f8356a-0df8-407a-ade3-10ac87e02de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.348720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b9f8356a-0df8-407a-ade3-10ac87e02de7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:42.349899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b9f8356a-0df8-407a-ade3-10ac87e02de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:42.421628 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 107/227] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:42.523897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fe52c-0fcc-44be-8ea1-fc1d5853392c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.524094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fe52c-0fcc-44be-8ea1-fc1d5853392c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.524577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079fe52c-0fcc-44be-8ea1-fc1d5853392c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.525920 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 121/228] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:42.531145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c8acf-cedf-4f76-a881-1e5682d0cd10 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.531429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c8acf-cedf-4f76-a881-1e5682d0cd10 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.531698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c8acf-cedf-4f76-a881-1e5682d0cd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.540533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:42.551483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2b0c8acf-cedf-4f76-a881-1e5682d0cd10 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:42.552688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2b0c8acf-cedf-4f76-a881-1e5682d0cd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:42.595840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fe52c-0fcc-44be-8ea1-fc1d5853392c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:42.620660 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 108/229] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:42.650130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:42.698270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c8acf-cedf-4f76-a881-1e5682d0cd10 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:42.707603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079fe52c-0fcc-44be-8ea1-fc1d5853392c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:42.747476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:42.748104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.748419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.748697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.759264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:42.764049 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:42.765060 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:42.766233 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 122/230] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:42.770794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:42.808174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c8acf-cedf-4f76-a881-1e5682d0cd10 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:42.844988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:42.845621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.845983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.846159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.847322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:42.849728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:42.851636 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 109/231] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:42.857236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:42.859872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:42.881726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1ef67-5791-4fa3-8fb3-e720796370dd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:42.885392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:42.918913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:42.919509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.919782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.920056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.920744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:42.921043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:42.922395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:42.930120 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 123/232] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:42.930883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:42.957715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5543befa-8a6f-4da5-b242-0f2f540eb968 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:42.991443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:42.992075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:42.992382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:42.992658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] PATH_INFO: `/v3/groups` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:42.993282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:42.994723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:43.011753 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 110/233] 10.222.0.39 () {60 vars in 1134 bytes} [Sun Oct 23 00:42:42 2022] POST /identity/v3/groups => generated 229 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:42:43.015020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:43.031004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1130a2b3-5f21-4be9-a632-63c669f20333 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:43.069608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:43.071170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:43.071170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:43.071380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:43.082989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:43.089274 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:43.090232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:43.091015 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 124/234] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:43.096371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:43.101178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:43.129587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:43.148998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8a9724-f064-4eef-b7fb-d81a40288674 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:43.189261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:43.189945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:43.190187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:43.190454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] PATH_INFO: `/v3/users/6e42dc501c784ae299adb6d02ea6d4d8` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:43.206951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] RBAC: Authorizing `identity:get_user(user_id=6e42dc501c784ae299adb6d02ea6d4d8)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:43.208488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:43.210596 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 111/235] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:43 2022] GET /identity/v3/users/6e42dc501c784ae299adb6d02ea6d4d8 => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:43.215861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:43.216457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b75cbe-53ea-41a6-8f63-1ba6f982d1b0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:43.254188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:43.254788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:43.255123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:43.255385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:43.256048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:43.256668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:43.258623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:43.268027 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 125/236] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:43.273593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:43.326176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f18db8-3353-4eb5-af1f-3f5145911dd5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:43.349416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:43.357825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:43.366697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:43.368063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:43.368565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:43.368971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:43.369835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] RBAC: Adding query filter params (role.id=36a32b37790246199c98277b5c065d58, user.id=6e42dc501c784ae299adb6d02ea6d4d8, scope.system=all) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:43.370280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:43.370483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8ea855-8291-4d0c-9fdc-2ffe442c1c26 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:43.372016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:43.377562 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 112/237] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Oct 23 00:42:43 2022] GET /identity/v3/role_assignments?user.id=6e42dc501c784ae299adb6d02ea6d4d8&scope.system=all&role.id=36a32b37790246199c98277b5c065d58 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:43.407262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed606ff-4f68-4f4f-8702-f8103d6b2d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:43.407912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed606ff-4f68-4f4f-8702-f8103d6b2d39 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:43.408191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed606ff-4f68-4f4f-8702-f8103d6b2d39 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:43.408492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed606ff-4f68-4f4f-8702-f8103d6b2d39 None admin] PATH_INFO: `/v3/users/cffd6287003746e09944f5d491a9fed5` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:43.409665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed606ff-4f68-4f4f-8702-f8103d6b2d39 None admin] RBAC: Authorizing `identity:get_user(user_id=cffd6287003746e09944f5d491a9fed5)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:43.411129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed606ff-4f68-4f4f-8702-f8103d6b2d39 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:43.413021 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 126/238] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:43 2022] GET /identity/v3/users/cffd6287003746e09944f5d491a9fed5 => generated 293 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:43.418895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:43.530883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31693ac-6ff5-44e1-a20a-067726b0bf19 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:43.567955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec0a43-3937-4d63-a6b9-02fc042f329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:43.568644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0a43-3937-4d63-a6b9-02fc042f329c None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:43.568925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0a43-3937-4d63-a6b9-02fc042f329c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:43.569196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec0a43-3937-4d63-a6b9-02fc042f329c None admin] PATH_INFO: `/v3/system/users/cffd6287003746e09944f5d491a9fed5/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:43.571177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec0a43-3937-4d63-a6b9-02fc042f329c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cffd6287003746e09944f5d491a9fed5, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:43.572578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec0a43-3937-4d63-a6b9-02fc042f329c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:43.582950 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 113/239] 10.222.0.39 () {58 vars in 1334 bytes} [Sun Oct 23 00:42:43 2022] PUT /identity/v3/system/users/cffd6287003746e09944f5d491a9fed5/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:42:45.200653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5ec50-afc9-4ab4-944d-5ed88c0dec68 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.201134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5ec50-afc9-4ab4-944d-5ed88c0dec68 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.201134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a5ec50-afc9-4ab4-944d-5ed88c0dec68 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.202622 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 127/240] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:45.207572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff17c53-4d24-441d-98c8-85eea232bbd3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.207943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff17c53-4d24-441d-98c8-85eea232bbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.208119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff17c53-4d24-441d-98c8-85eea232bbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.225939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-eff17c53-4d24-441d-98c8-85eea232bbd3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:45.227634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eff17c53-4d24-441d-98c8-85eea232bbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:45.272669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:45.298201 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 114/241] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:45.527487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9fb8-c674-4018-9038-142f40aae89d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.527723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9fb8-c674-4018-9038-142f40aae89d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.528158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba9fb8-c674-4018-9038-142f40aae89d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.532359 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 128/242] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:45.534964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ba41-0310-4c27-b05c-16ad084033f1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.535244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ba41-0310-4c27-b05c-16ad084033f1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.535515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c6ba41-0310-4c27-b05c-16ad084033f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.553996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-96c6ba41-0310-4c27-b05c-16ad084033f1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:45.555143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-96c6ba41-0310-4c27-b05c-16ad084033f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:45.567352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba9fb8-c674-4018-9038-142f40aae89d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:45.627667 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 115/243] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:45.681421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba9fb8-c674-4018-9038-142f40aae89d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:45.715198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:45.715820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.716091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.716385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.717462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:45.718881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:45.720950 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 129/244] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:42:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:45.723948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029afa2-e29c-4b24-b3ec-7ddd32d6301e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.724241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029afa2-e29c-4b24-b3ec-7ddd32d6301e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.724548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029afa2-e29c-4b24-b3ec-7ddd32d6301e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.725901 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 116/245] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:45.726827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:45.731290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9eeb0-052c-4b63-a1e7-5a9cc78251d3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.731576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9eeb0-052c-4b63-a1e7-5a9cc78251d3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.731852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9eeb0-052c-4b63-a1e7-5a9cc78251d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.749529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-87d9eeb0-052c-4b63-a1e7-5a9cc78251d3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:45.750539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-87d9eeb0-052c-4b63-a1e7-5a9cc78251d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:45.753025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:45.825761 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 117/246] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:45.836506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21efdd88-0d1b-412c-b8ac-8b978b3a3dad None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:45.870342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:45.870953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:45.871233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:45.871507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] PATH_INFO: `/v3/groups` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:45.872134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:45.873601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:45.892461 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 130/247] 10.222.0.39 () {60 vars in 1134 bytes} [Sun Oct 23 00:42:45 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:42:45.896215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9eeb0-052c-4b63-a1e7-5a9cc78251d3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:45.942103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:46.008048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9eeb0-052c-4b63-a1e7-5a9cc78251d3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.044113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.044836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.045113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.045376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.056230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:46.061106 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:46.061989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:46.062793 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 118/248] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:46.069137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:46.099852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:46.176606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:46.201274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76e10f4-8331-418f-b9e8-78c0fd13aaac None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.212852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:46.218472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44276fb-3ab4-454a-a58f-6b1efc98f85f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.241006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.242139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.242435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.242708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.243376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] RBAC: Adding query filter params (name=reader) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:46.243682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:46.246585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:46.255854 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 131/249] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:46.258454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.259092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.259368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.259636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.262238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:46.273627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:46.278744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:46.279607 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:46.280484 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 119/250] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:46.286504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:46.391125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2a3847-faea-49bd-ac94-f206cdf5a4ac None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.422018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0c258f-7e0a-4a58-9bf6-0636e1801e40 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.428469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40762c15-efce-4585-a981-1949a404050e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.429855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40762c15-efce-4585-a981-1949a404050e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.430248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40762c15-efce-4585-a981-1949a404050e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.430615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40762c15-efce-4585-a981-1949a404050e None admin] PATH_INFO: `/v3/users/6e42dc501c784ae299adb6d02ea6d4d8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.431867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40762c15-efce-4585-a981-1949a404050e None admin] RBAC: Authorizing `identity:get_user(user_id=6e42dc501c784ae299adb6d02ea6d4d8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:46.433450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40762c15-efce-4585-a981-1949a404050e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:46.435325 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 132/251] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:46 2022] GET /identity/v3/users/6e42dc501c784ae299adb6d02ea6d4d8 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:46.441319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40762c15-efce-4585-a981-1949a404050e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:46.460924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:46.473133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.473748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.474014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.474279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.474952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:46.475247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:46.476709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:46.487518 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 120/252] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:46.493874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:46.525594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:46.570921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40762c15-efce-4585-a981-1949a404050e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.613192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.615509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.616192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.617593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] PATH_INFO: `/v3/system/users/6e42dc501c784ae299adb6d02ea6d4d8/roles/36a32b37790246199c98277b5c065d58` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.620063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6e42dc501c784ae299adb6d02ea6d4d8, role_id=36a32b37790246199c98277b5c065d58)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:46.621922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:46.627774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a82b3f-b127-43d7-82a2-4974f40e44f8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.636434 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 133/253] 10.222.0.39 () {58 vars in 1334 bytes} [Sun Oct 23 00:42:46 2022] PUT /identity/v3/system/users/6e42dc501c784ae299adb6d02ea6d4d8/roles/36a32b37790246199c98277b5c065d58 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:46.669176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0062c4f9-7c40-480e-89db-9eaa63d7f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.669754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062c4f9-7c40-480e-89db-9eaa63d7f341 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.670029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062c4f9-7c40-480e-89db-9eaa63d7f341 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.670298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0062c4f9-7c40-480e-89db-9eaa63d7f341 None admin] PATH_INFO: `/v3/users/cffd6287003746e09944f5d491a9fed5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.671474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062c4f9-7c40-480e-89db-9eaa63d7f341 None admin] RBAC: Authorizing `identity:get_user(user_id=cffd6287003746e09944f5d491a9fed5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:46.673059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0062c4f9-7c40-480e-89db-9eaa63d7f341 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:46.676130 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 121/254] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:46 2022] GET /identity/v3/users/cffd6287003746e09944f5d491a9fed5 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:46.682899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:46.780150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:46.806144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa4a25-c268-4e17-a7f0-acd3d5a62d44 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:46.846388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0327b43a-2c50-4435-8486-9fc5cfb27160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:46.847005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327b43a-2c50-4435-8486-9fc5cfb27160 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:46.847278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327b43a-2c50-4435-8486-9fc5cfb27160 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:46.847545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0327b43a-2c50-4435-8486-9fc5cfb27160 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:46.848274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327b43a-2c50-4435-8486-9fc5cfb27160 None admin] RBAC: Adding query filter params (role.id=f6e93371f84c47b0b0ecf09d4467080d, user.id=cffd6287003746e09944f5d491a9fed5, scope.system=all) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:46.848621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327b43a-2c50-4435-8486-9fc5cfb27160 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:46.850195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0327b43a-2c50-4435-8486-9fc5cfb27160 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:46.863853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 134/255] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Oct 23 00:42:46 2022] GET /identity/v3/role_assignments?user.id=cffd6287003746e09944f5d491a9fed5&scope.system=all&role.id=f6e93371f84c47b0b0ecf09d4467080d => generated 510 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:48.339921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06becd5-1a52-4327-8f92-5a2cd567817e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:48.340456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06becd5-1a52-4327-8f92-5a2cd567817e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:48.340456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06becd5-1a52-4327-8f92-5a2cd567817e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:48.341983 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 122/256] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:48.347661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d207-5b94-4090-9f37-7b6411f4b1c7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:48.348547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d207-5b94-4090-9f37-7b6411f4b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:48.349031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2925d207-5b94-4090-9f37-7b6411f4b1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:48.368150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2925d207-5b94-4090-9f37-7b6411f4b1c7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:48.369671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2925d207-5b94-4090-9f37-7b6411f4b1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:48.441368 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 135/257] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 00:42:48.575688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8924-7f3d-40d6-b52e-3ed522737c42 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:48.576010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8924-7f3d-40d6-b52e-3ed522737c42 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:48.576225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fced8924-7f3d-40d6-b52e-3ed522737c42 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:48.577600 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 123/258] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:48.582948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e79d-8b42-42df-a8c4-8a89f6df65dd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:48.583233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e79d-8b42-42df-a8c4-8a89f6df65dd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:48.583503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e5e79d-8b42-42df-a8c4-8a89f6df65dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:48.609354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b0e5e79d-8b42-42df-a8c4-8a89f6df65dd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:48.610873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b0e5e79d-8b42-42df-a8c4-8a89f6df65dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:48.621868 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 136/259] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 00:42:49.038546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8924-7f3d-40d6-b52e-3ed522737c42 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.065624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fced8924-7f3d-40d6-b52e-3ed522737c42 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.076969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.077684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.077967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.078231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.097203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:49.102086 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:49.102949 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:49.103733 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:49.104825 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 124/260] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 65 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:49.109839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5e79d-8b42-42df-a8c4-8a89f6df65dd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.119148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e5e79d-8b42-42df-a8c4-8a89f6df65dd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.132779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.133915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.134392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.139659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.141385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:49.141898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.143972 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:42:49.143972 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:42:49.143972 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:42:49.145274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.155448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 137/261] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:49.160874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.181665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eea194-fdb9-4029-9a3a-48339cddde40 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.193047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.193711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.193984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.194248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.202176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.203372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.205219 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:42:49.205219 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:42:49.205219 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:42:49.206476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.213645 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 125/262] 10.222.0.39 () {56 vars in 1201 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:49.214990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.221718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cddd93-5d03-4990-9287-f3299ddff974 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.226364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc9a34-97f3-46f8-a253-0ae91b484f79 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.230828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.231531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.231846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.232119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.234259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.234608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.235484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.235959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.236462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.236566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.238446 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 126/263] 10.222.0.39 () {56 vars in 1207 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:49.244743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.255422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82367a7-d982-45b9-bcd5-f64effabab5b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.265183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.265922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.266238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.266509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/groups/150041a347474b7f898024889f965044/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.268449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76, group_id=150041a347474b7f898024889f965044, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.269488 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:42:49.269488 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:42:49.269488 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:42:49.273267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:49.277980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:49.285336 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:49.286226 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:49.287015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:49.290170 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 138/264] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 89 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:49.296380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.321470 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 127/265] 10.222.0.39 () {58 vars in 1463 bytes} [Sun Oct 23 00:42:49 2022] PUT /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/groups/150041a347474b7f898024889f965044/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:42:49.323084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe48e21c-16b4-4b61-96e2-c0f531d9dc81 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.433095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.434594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.435078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.435561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.436525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:49.437043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.438948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.441017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:49.459157 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 139/266] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:49.464876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.494399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41bb059-bb29-4a2a-9a3b-b0e53a269053 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.501241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.502160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.502588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.502998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.504290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.506705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.513591 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 128/267] 10.222.0.39 () {56 vars in 1201 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:49.515171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.536081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a07dff-ba9c-4bfc-8210-63d64617f667 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.544210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e070c-9289-46df-a168-97e64d700985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.544979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e070c-9289-46df-a168-97e64d700985 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.545361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e070c-9289-46df-a168-97e64d700985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.545745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e070c-9289-46df-a168-97e64d700985 admin admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.553207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e070c-9289-46df-a168-97e64d700985 admin admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.556014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e070c-9289-46df-a168-97e64d700985 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.558236 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 140/268] 10.222.0.39 () {56 vars in 1207 bytes} [Sun Oct 23 00:42:49 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:49.564038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:49.577615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d83ed4-a38f-4d13-a1de-738a814b4539 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:49.584990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db82482-f923-433a-a6fb-2b3034b2db95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:49.586087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db82482-f923-433a-a6fb-2b3034b2db95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.586534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db82482-f923-433a-a6fb-2b3034b2db95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.586987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db82482-f923-433a-a6fb-2b3034b2db95 admin admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/groups/150041a347474b7f898024889f965044/roles/633fd4ad499a4f62a5a2cbac0eb861ce` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.589637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db82482-f923-433a-a6fb-2b3034b2db95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76, group_id=150041a347474b7f898024889f965044, role_id=633fd4ad499a4f62a5a2cbac0eb861ce)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:49.592724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db82482-f923-433a-a6fb-2b3034b2db95 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:49.624862 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 129/269] 10.222.0.39 () {58 vars in 1463 bytes} [Sun Oct 23 00:42:49 2022] PUT /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76/groups/150041a347474b7f898024889f965044/roles/633fd4ad499a4f62a5a2cbac0eb861ce => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:42:49.728063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c3cf0-5256-4e90-a233-cab51bba227d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.728418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c3cf0-5256-4e90-a233-cab51bba227d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.728680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c3cf0-5256-4e90-a233-cab51bba227d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.730317 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 141/270] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:49.736247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3974f0-5f46-4804-869b-d5f713cc14cc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.736927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3974f0-5f46-4804-869b-d5f713cc14cc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.737580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3974f0-5f46-4804-869b-d5f713cc14cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.763166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8c3974f0-5f46-4804-869b-d5f713cc14cc None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:49.764837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8c3974f0-5f46-4804-869b-d5f713cc14cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:49.804792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:49.853164 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 130/271] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:49.903161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ee93c-c86f-40a5-9262-0523c1ad7d5b None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.903735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ee93c-c86f-40a5-9262-0523c1ad7d5b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.904189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ee93c-c86f-40a5-9262-0523c1ad7d5b None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.906095 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 142/272] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:49.911760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daefac-50e2-4ec7-a3dd-73269946793e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:49.912038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daefac-50e2-4ec7-a3dd-73269946793e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:49.912327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daefac-50e2-4ec7-a3dd-73269946793e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:49.932742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b9daefac-50e2-4ec7-a3dd-73269946793e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:49.934306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b9daefac-50e2-4ec7-a3dd-73269946793e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:50.012090 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 131/273] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 00:42:50.158051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ee93c-c86f-40a5-9262-0523c1ad7d5b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.293215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ee93c-c86f-40a5-9262-0523c1ad7d5b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.338819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.339839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.340358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.340836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.355257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:50.360412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:50.361310 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 00:42:50.362122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:50.364369 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 143/274] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:50.370977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daefac-50e2-4ec7-a3dd-73269946793e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.426103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.438998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9fd26-c416-4691-9b48-da4855d25025 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.443370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.444268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.444821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.445290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.458045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:50.463331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:50.464177 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:50.464989 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:50.467514 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 144/275] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 42 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:50.473754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.487069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f122bd-40d8-4593-a0b0-27e23b838352 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.492570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.493469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.494032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.494509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.495367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:50.495876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:50.497714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:50.512198 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 145/276] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:50.513442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.515519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daefac-50e2-4ec7-a3dd-73269946793e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.524022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d65d8-3dd6-4ac5-9df9-9e7e0414d453 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.528777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.529624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.530096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.530562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.531870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:50.533126 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:42:50.533126 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:42:50.533126 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:42:50.534657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:50.534834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:50.537105 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 146/277] 10.222.0.39 () {56 vars in 1201 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:50.543263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.554627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d414b1d-2139-419e-b7ea-6a6461fb5580 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.559587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.560544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.561085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.561548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.563198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:50.565851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:50.568245 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 147/278] 10.222.0.39 () {56 vars in 1207 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:50.577497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.581009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.581650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.581932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.582205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.582885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:50.583362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:50.584727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:50.589670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2de987-e4b3-4126-a2a8-c0e78cc1e544 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.595572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.596271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.596658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.596935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891/groups/150041a347474b7f898024889f965044/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.599327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abcb12b59954bc4b95706027f575891, group_id=150041a347474b7f898024889f965044, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:50.599989 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 132/279] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:50.600445 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:42:50.600445 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:42:50.600445 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:42:50.603085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:50.606303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.632851 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 148/280] 10.222.0.39 () {58 vars in 1463 bytes} [Sun Oct 23 00:42:50 2022] PUT /identity/v3/projects/9abcb12b59954bc4b95706027f575891/groups/150041a347474b7f898024889f965044/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:42:50.763320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47de90-7ccd-426a-a8e0-7346668b2a39 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:50.806784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b80c43e-805e-4e84-83b3-f0c70dfe0e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:50.807431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80c43e-805e-4e84-83b3-f0c70dfe0e2e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:50.807704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80c43e-805e-4e84-83b3-f0c70dfe0e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:50.807970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b80c43e-805e-4e84-83b3-f0c70dfe0e2e None admin] PATH_INFO: `/v3/users/6e42dc501c784ae299adb6d02ea6d4d8` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:50.809179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80c43e-805e-4e84-83b3-f0c70dfe0e2e None admin] RBAC: Authorizing `identity:get_user(user_id=6e42dc501c784ae299adb6d02ea6d4d8)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:50.810640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b80c43e-805e-4e84-83b3-f0c70dfe0e2e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:50.812218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 133/281] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/users/6e42dc501c784ae299adb6d02ea6d4d8 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:50.819924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:50.847995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:50.965144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8399838d-b1c7-42ba-9af6-ee98ac6daf34 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:51.004771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de5abe-1c13-4d04-8747-cbe03046038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:51.005392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5abe-1c13-4d04-8747-cbe03046038b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:51.005671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5abe-1c13-4d04-8747-cbe03046038b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:51.005944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de5abe-1c13-4d04-8747-cbe03046038b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:51.006953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5abe-1c13-4d04-8747-cbe03046038b None admin] RBAC: Adding query filter params (role.id=36a32b37790246199c98277b5c065d58, user.id=6e42dc501c784ae299adb6d02ea6d4d8, scope.system=all) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:51.007817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5abe-1c13-4d04-8747-cbe03046038b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:51.009321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de5abe-1c13-4d04-8747-cbe03046038b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:51.027105 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 149/282] 10.222.0.39 () {56 vars in 1308 bytes} [Sun Oct 23 00:42:50 2022] GET /identity/v3/role_assignments?user.id=6e42dc501c784ae299adb6d02ea6d4d8&scope.system=all&role.id=36a32b37790246199c98277b5c065d58 => generated 510 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:51.361230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97739a8-382c-4ee2-90b2-86f0d02e9c3a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:51.361592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97739a8-382c-4ee2-90b2-86f0d02e9c3a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:51.361838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97739a8-382c-4ee2-90b2-86f0d02e9c3a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:51.363186 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 134/283] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:51.368512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb493f3-a934-471e-a823-a18decaf195d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:51.368792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb493f3-a934-471e-a823-a18decaf195d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:51.369075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb493f3-a934-471e-a823-a18decaf195d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:51.393494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-edb493f3-a934-471e-a823-a18decaf195d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:51.394696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-edb493f3-a934-471e-a823-a18decaf195d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:51.461430 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 150/284] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 00:42:51.726676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5fdcf-5605-40ee-b3ca-9fc3ae3f57b9 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:51.727172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5fdcf-5605-40ee-b3ca-9fc3ae3f57b9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:51.727426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5fdcf-5605-40ee-b3ca-9fc3ae3f57b9 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:51.728976 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 135/285] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:51.735042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc8504-4c4f-496c-8815-38d5bd90e59f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:51.735332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc8504-4c4f-496c-8815-38d5bd90e59f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:51.735606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fc8504-4c4f-496c-8815-38d5bd90e59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:51.755695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c1fc8504-4c4f-496c-8815-38d5bd90e59f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:51.757208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c1fc8504-4c4f-496c-8815-38d5bd90e59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:51.766926 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 151/286] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 00:42:52.004207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5fdcf-5605-40ee-b3ca-9fc3ae3f57b9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.018862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5fdcf-5605-40ee-b3ca-9fc3ae3f57b9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.025732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.026726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.027219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.027813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.041727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:52.046805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:52.047663 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:52.048479 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:52.050659 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 136/287] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 47 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:52.056580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc8504-4c4f-496c-8815-38d5bd90e59f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.072346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fc8504-4c4f-496c-8815-38d5bd90e59f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.079314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.079983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.080268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.080562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.081241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:52.081541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.083150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.088498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:52.099718 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 152/288] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:52.103005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.104911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.115961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7489e9-33ac-4cff-bf7a-08f3fa339286 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.118570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d19b9-e3b4-44bd-bb0c-e52d4c08f934 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.120871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.121511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.121790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.122062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.123168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.125241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.127207 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 153/289] 10.222.0.39 () {56 vars in 1201 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:52.127562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.128447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.128970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.129432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.134563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.146307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54afd82-03da-4418-ae4d-f75786bee280 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.149347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:52.151834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.152555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.152838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.153168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.154350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:52.155797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:52.156645 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:52.157421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:52.157999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.162625 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 154/290] 10.222.0.39 () {56 vars in 1207 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:52.163778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.164603 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 137/291] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 62 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:52.170320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.174764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e80f596-c446-4d11-832d-510e9e7e1114 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.180015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.181025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.181349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.181685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891/groups/150041a347474b7f898024889f965044/roles/633fd4ad499a4f62a5a2cbac0eb861ce` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.186025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9abcb12b59954bc4b95706027f575891, group_id=150041a347474b7f898024889f965044, role_id=633fd4ad499a4f62a5a2cbac0eb861ce)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.188522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a610deb-c5e7-4ed0-b9cc-9f5322daa64b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.189003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.197860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.198550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.198816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.199092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.199870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:52.200186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.207932 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 155/292] 10.222.0.39 () {58 vars in 1463 bytes} [Sun Oct 23 00:42:52 2022] PUT /identity/v3/projects/9abcb12b59954bc4b95706027f575891/groups/150041a347474b7f898024889f965044/roles/633fd4ad499a4f62a5a2cbac0eb861ce => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:42:52.208214 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:42:52.208214 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:42:52.208214 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:42:52.209192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.220273 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 138/293] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:52.226119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.300951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca421f8-9fa9-4878-946c-8dbb36356c1f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.306200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.306845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.307126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.307404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] PATH_INFO: `/v3/groups/0edb1a6bf3da4bc5a666884fb73a8c19` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.314172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] RBAC: Authorizing `identity:get_group(group_id=0edb1a6bf3da4bc5a666884fb73a8c19)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.316288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.318572 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 156/294] 10.222.0.39 () {56 vars in 1201 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/groups/0edb1a6bf3da4bc5a666884fb73a8c19 => generated 229 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:52.325070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.335450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1cdff-cae5-4b45-baf6-91bb3d2ed2a2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.340340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b52c2a-7b23-4b60-a2ef-c401003b37ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.341063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b52c2a-7b23-4b60-a2ef-c401003b37ce admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.341352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b52c2a-7b23-4b60-a2ef-c401003b37ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.341628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b52c2a-7b23-4b60-a2ef-c401003b37ce admin admin] PATH_INFO: `/v3/projects/ee25af34c3554966b7a70b6f9a37af9a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.342894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b52c2a-7b23-4b60-a2ef-c401003b37ce admin admin] RBAC: Authorizing `identity:get_project(project_id=ee25af34c3554966b7a70b6f9a37af9a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.344888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b52c2a-7b23-4b60-a2ef-c401003b37ce admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.347120 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 139/295] 10.222.0.39 () {56 vars in 1207 bytes} [Sun Oct 23 00:42:52 2022] GET /identity/v3/projects/ee25af34c3554966b7a70b6f9a37af9a => generated 344 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:52.353057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:52.363786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ad4d8-6a65-47c0-b0b3-06725e4b488b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:52.369235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53e19d6-563b-4cc7-9c8c-1d007b6451ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:52.369972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e19d6-563b-4cc7-9c8c-1d007b6451ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.370256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e19d6-563b-4cc7-9c8c-1d007b6451ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.370535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53e19d6-563b-4cc7-9c8c-1d007b6451ba admin admin] PATH_INFO: `/v3/projects/ee25af34c3554966b7a70b6f9a37af9a/groups/0edb1a6bf3da4bc5a666884fb73a8c19/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.372623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e19d6-563b-4cc7-9c8c-1d007b6451ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee25af34c3554966b7a70b6f9a37af9a, group_id=0edb1a6bf3da4bc5a666884fb73a8c19, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:52.375416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53e19d6-563b-4cc7-9c8c-1d007b6451ba admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:52.396252 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 157/296] 10.222.0.39 () {58 vars in 1463 bytes} [Sun Oct 23 00:42:52 2022] PUT /identity/v3/projects/ee25af34c3554966b7a70b6f9a37af9a/groups/0edb1a6bf3da4bc5a666884fb73a8c19/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:42:52.673730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1df4c-9586-46c6-9925-2602bfd4a45b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.674078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1df4c-9586-46c6-9925-2602bfd4a45b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.674241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1df4c-9586-46c6-9925-2602bfd4a45b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.675784 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 140/297] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:52.681751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20c2d-3020-48a3-b507-58f093dda612 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.682279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20c2d-3020-48a3-b507-58f093dda612 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.682279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a20c2d-3020-48a3-b507-58f093dda612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.700601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-00a20c2d-3020-48a3-b507-58f093dda612 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:52.701783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-00a20c2d-3020-48a3-b507-58f093dda612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:52.774184 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 158/298] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:52.941216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb4cbe-a18f-4946-ace8-fbed615d8fa8 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.942142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb4cbe-a18f-4946-ace8-fbed615d8fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.942621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb4cbe-a18f-4946-ace8-fbed615d8fa8 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.944161 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 141/299] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:52.949913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e16e7-6a54-477b-8727-64846b0c3831 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:52.950468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e16e7-6a54-477b-8727-64846b0c3831 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:52.950930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e16e7-6a54-477b-8727-64846b0c3831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:52.969874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-424e16e7-6a54-477b-8727-64846b0c3831 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:52.971381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-424e16e7-6a54-477b-8727-64846b0c3831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:53.019243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:53.055563 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 159/300] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:42:53.274927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb4cbe-a18f-4946-ace8-fbed615d8fa8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:53.278724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:53.409303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb4cbe-a18f-4946-ace8-fbed615d8fa8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:53.450695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:53.451328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:53.451603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:53.451870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:53.465436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:53.470679 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:53.471583 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:53.472409 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 142/301] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:53.478318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e16e7-6a54-477b-8727-64846b0c3831 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:53.596906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e16e7-6a54-477b-8727-64846b0c3831 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:53.636572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:53.637183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:53.637456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:53.637722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:53.638383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:53.638677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:53.640080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:53.648675 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 160/302] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:53.654500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:53.687205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:53.743629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:53.773481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:53.824257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09bf089-fb92-492c-b336-a448290d8085 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:53.868683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d280f-9ef7-47fe-b3c6-6d2f8d9a2c20 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:53.871114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1db84fc-4a79-45da-9b61-a639fef15b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:53.871731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db84fc-4a79-45da-9b61-a639fef15b1d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:53.872003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db84fc-4a79-45da-9b61-a639fef15b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:53.872262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1db84fc-4a79-45da-9b61-a639fef15b1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:53.914513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:53.915147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:53.915423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:53.915687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:53.941298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:53.951291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:53.952179 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:42:53.953000 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 161/303] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:42:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:53.982802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:54.020659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db84fc-4a79-45da-9b61-a639fef15b1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:54.022155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1db84fc-4a79-45da-9b61-a639fef15b1d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:54.143321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:54.157243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:54.168041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb04e16-e14d-4ddf-9f96-52f7fef23ef1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:54.192980 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 143/304] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:54.195082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f11e51-c44e-4518-9d54-a1ac342220ec None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:54.195575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f11e51-c44e-4518-9d54-a1ac342220ec None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:54.196040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f11e51-c44e-4518-9d54-a1ac342220ec None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:54.197643 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 144/305] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:54.203570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25660d-0e50-4fb9-9855-8eb6abc7a493 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:54.204366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25660d-0e50-4fb9-9855-8eb6abc7a493 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:54.204844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b25660d-0e50-4fb9-9855-8eb6abc7a493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:54.217439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:54.218133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:54.218499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:54.218832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:54.219566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:54.219926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:54.221912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:54.227529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1b25660d-0e50-4fb9-9855-8eb6abc7a493 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:54.228932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b25660d-0e50-4fb9-9855-8eb6abc7a493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:54.233066 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 162/306] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:42:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:54.233806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:54.315338 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 145/307] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:54.316320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25660d-0e50-4fb9-9855-8eb6abc7a493 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:54.388487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8140f-f6f9-4de3-a5a4-5253c9ae5484 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:54.443013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:54.444431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:54.445332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:54.445603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:54.450824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:54.452335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:54.454040 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 163/308] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:54 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:54.459554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:54.488061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:54.501420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b25660d-0e50-4fb9-9855-8eb6abc7a493 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:54.504517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:54.589031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:54.589669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:54.589948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:54.590221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:54.601985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5814c8-127e-46a3-8efa-ac22de7be971 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:54.663940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:54.665750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:54.666036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:54.666312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:54.667988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:54.669459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:54.671044 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 164/309] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Oct 23 00:42:54 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:54.678433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:54.714175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:54.715175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:54.794859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:54.809025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16899ff4-6638-4071-8485-c7c973052c29 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:54.849401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:54.853550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:54.854481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:54.854964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:54.855186 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 146/310] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:54.856129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:54.857150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] RBAC: Adding query filter params (group.id=150041a347474b7f898024889f965044, role.id=f6e93371f84c47b0b0ecf09d4467080d, scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:54.857277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:54.857772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:54.859493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:54.866230 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 165/311] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Oct 23 00:42:54 2022] GET /identity/v3/role_assignments?group.id=150041a347474b7f898024889f965044&scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76&role.id=f6e93371f84c47b0b0ecf09d4467080d => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:54.868820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.003951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edad18-1df7-492a-a8dd-5c57307dd7dc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.049767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ba5746-2e79-4347-a302-19a219123d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.051515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba5746-2e79-4347-a302-19a219123d60 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.052216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba5746-2e79-4347-a302-19a219123d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.056243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba5746-2e79-4347-a302-19a219123d60 None admin] PATH_INFO: `/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.058005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba5746-2e79-4347-a302-19a219123d60 None admin] RBAC: Authorizing `identity:get_project(project_id=84e67cbc8f2f4aa4aad8a3e24a095a76)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:55.059845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ba5746-2e79-4347-a302-19a219123d60 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:55.061980 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 166/312] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:54 2022] GET /identity/v3/projects/84e67cbc8f2f4aa4aad8a3e24a095a76 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:55.063651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ac789-ba9c-4ebc-9e73-76b1f1207b61 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.064175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ac789-ba9c-4ebc-9e73-76b1f1207b61 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.064714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ac789-ba9c-4ebc-9e73-76b1f1207b61 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.066381 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 167/313] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:55.069859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ac789-ba9c-4ebc-9e73-76b1f1207b61 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.079958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc6108-02d1-4730-9e5f-a205ef13c0c4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.115598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:55.145386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812ceedc-25d9-4762-83d1-0f78cfdae906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.146201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ceedc-25d9-4762-83d1-0f78cfdae906 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.146478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ceedc-25d9-4762-83d1-0f78cfdae906 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.146751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ceedc-25d9-4762-83d1-0f78cfdae906 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.159787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812ceedc-25d9-4762-83d1-0f78cfdae906 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-812ceedc-25d9-4762-83d1-0f78cfdae906 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:55.166040 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:55.166895 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:55.168059 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 147/314] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:55.168832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a86f9f-7872-4357-beb5-8e2d63cb6cc7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.168832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a86f9f-7872-4357-beb5-8e2d63cb6cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.168962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a86f9f-7872-4357-beb5-8e2d63cb6cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.194347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-29a86f9f-7872-4357-beb5-8e2d63cb6cc7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:55.196241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-29a86f9f-7872-4357-beb5-8e2d63cb6cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:55.224043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ac789-ba9c-4ebc-9e73-76b1f1207b61 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.254675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:55.264410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79528ed-c5b9-4839-b078-3ef6ae85292f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.265051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79528ed-c5b9-4839-b078-3ef6ae85292f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.265339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79528ed-c5b9-4839-b078-3ef6ae85292f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.265613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79528ed-c5b9-4839-b078-3ef6ae85292f None admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.269611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79528ed-c5b9-4839-b078-3ef6ae85292f None admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:55.269611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79528ed-c5b9-4839-b078-3ef6ae85292f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:55.270023 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 168/315] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Oct 23 00:42:55 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:55.271154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a261e4-8dc5-4c76-88c8-a9ddc45053dd None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.271556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a261e4-8dc5-4c76-88c8-a9ddc45053dd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.271832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a261e4-8dc5-4c76-88c8-a9ddc45053dd None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.273633 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 169/316] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:42:55.274332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a261e4-8dc5-4c76-88c8-a9ddc45053dd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.281701 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 148/317] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:55.282794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f99c7-59a1-4283-99b0-bac2fcb5a8d0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.283202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f99c7-59a1-4283-99b0-bac2fcb5a8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.283476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20f99c7-59a1-4283-99b0-bac2fcb5a8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.307503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e20f99c7-59a1-4283-99b0-bac2fcb5a8d0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:42:55.309304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e20f99c7-59a1-4283-99b0-bac2fcb5a8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:42:55.381776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:55.390875 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 149/318] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:42:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:42:55.391633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f99c7-59a1-4283-99b0-bac2fcb5a8d0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.396041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a261e4-8dc5-4c76-88c8-a9ddc45053dd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.432224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.433139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.433588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.434027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.434930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] RBAC: Adding query filter params (name=member) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:55.435400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:55.436957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:55.446761 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 170/319] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:42:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:55.454319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.514825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20f99c7-59a1-4283-99b0-bac2fcb5a8d0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.530613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:55.556000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.557806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.558088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.558355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.559085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] RBAC: Adding query filter params (group.id=150041a347474b7f898024889f965044, role.id=633fd4ad499a4f62a5a2cbac0eb861ce, scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:55.559456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:55.560857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:55.561649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb5580-8d19-44e0-9e49-d0899baffe06 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.567003 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 150/320] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Oct 23 00:42:55 2022] GET /identity/v3/role_assignments?group.id=150041a347474b7f898024889f965044&scope.project.id=84e67cbc8f2f4aa4aad8a3e24a095a76&role.id=633fd4ad499a4f62a5a2cbac0eb861ce => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:55.581932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.601961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.602584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.602856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.603178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.634254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:55.697650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:55.699480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:55.751694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3400ee7a-e2cf-4123-8314-5fffd059cce9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.800028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.801012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.801482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.801959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:55.813494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:55.820339 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:55.821210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:42:55.822007 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:55.824392 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 151/321] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:42:55.825433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.840608 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 171/322] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:55.841367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:55.875952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:55.905894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:55.953819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ef2f0-2000-4b8f-9370-bddbbab61bc3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.961392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e6715-3705-4888-86b6-47a70dcff0af None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:55.994668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:55.995300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:55.995576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:55.995846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.000038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.000697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.000978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.001247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.002079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:56.002377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.003737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.008907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:42:56.013703 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:42:56.014791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:42:56.015753 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:42:56.016530 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 172/323] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:56.016613 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 152/324] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:42:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:42:56.016689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.022813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.140421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.143287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a211207-aeac-4458-9354-8e553389b557 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.145399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e37345-9d74-4754-abd3-df686f4c24b6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.160347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.186011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.186736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.187019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.187292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.188448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.189801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.189947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.190419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.190699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.190969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.196347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:56.196495 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 173/325] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:56.196610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.196702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.196794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.203066 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 153/326] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:56.203952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.315223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc95ad-6f28-4ab6-a3c3-d973c069f35d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.334360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c839fc87-44cf-465d-b04b-0aeacc0dbe6c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.353741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.354370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.354634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.354906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.374478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.375388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.375815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.376080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.377275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.378702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.380265 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 154/327] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:56.382446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.416974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.417154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.441180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.441453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.505574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875846d5-12f4-4340-b400-e87ccec60f4b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.539368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.539974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.540252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.540552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.550121 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 174/328] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:56.550962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.611570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.613059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.630116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.640378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.649386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a40c4f1-9eb6-44c1-855c-e8563595550b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.683030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.683561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.683841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.684112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.684879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] RBAC: Adding query filter params (group.id=150041a347474b7f898024889f965044, role.id=f6e93371f84c47b0b0ecf09d4467080d, scope.project.id=9abcb12b59954bc4b95706027f575891) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:56.685185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.686604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.692177 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 175/329] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/role_assignments?group.id=150041a347474b7f898024889f965044&scope.project.id=9abcb12b59954bc4b95706027f575891&role.id=f6e93371f84c47b0b0ecf09d4467080d => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:56.692983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.717308 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 155/330] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:42:56.723228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.800749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb34369-2b2b-42cb-ad67-5d62b13d58f5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.840681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd85098-1850-46f6-ac34-e3310c43100e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:56.841012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.841623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.841898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.842167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] PATH_INFO: `/v3/projects/ee25af34c3554966b7a70b6f9a37af9a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.843322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] RBAC: Authorizing `identity:get_project(project_id=ee25af34c3554966b7a70b6f9a37af9a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.844797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.851576 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 176/331] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/projects/ee25af34c3554966b7a70b6f9a37af9a => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:56.852690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.869400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.877127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:56.886071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:56.886725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:56.886986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:56.887258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] PATH_INFO: `/v3/projects/9abcb12b59954bc4b95706027f575891` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:56.888996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] RBAC: Authorizing `identity:get_project(project_id=9abcb12b59954bc4b95706027f575891)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:56.890698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:56.892600 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 156/332] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/projects/9abcb12b59954bc4b95706027f575891 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:56.899033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:56.967860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ee28eb-3fdd-4890-b61b-55393fb167fc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:57.007444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:57.008252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:57.008871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:57.009161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] PATH_INFO: `/v3/groups/0edb1a6bf3da4bc5a666884fb73a8c19` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:57.010426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] RBAC: Authorizing `identity:get_group(group_id=0edb1a6bf3da4bc5a666884fb73a8c19)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:57.011332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf656cd9-9c80-441e-b597-9b65a87b13a8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:57.011863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:57.013471 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 177/333] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/groups/0edb1a6bf3da4bc5a666884fb73a8c19 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:57.019289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:57.053338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:57.054068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:57.054341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:57.054612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] PATH_INFO: `/v3/groups/150041a347474b7f898024889f965044` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:57.055722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] RBAC: Authorizing `identity:get_group(group_id=150041a347474b7f898024889f965044)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:57.057182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:57.058911 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 157/334] 10.222.0.39 () {56 vars in 1180 bytes} [Sun Oct 23 00:42:56 2022] GET /identity/v3/groups/150041a347474b7f898024889f965044 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:42:57.064816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:42:57.108786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:57.128494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d56c5-36bf-4699-b18b-3d41bf3995fa None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:57.134520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:42:57.162443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a134e1e-7ed0-4b89-afbc-8fd7e0bb8324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:57.163047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a134e1e-7ed0-4b89-afbc-8fd7e0bb8324 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:57.163351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a134e1e-7ed0-4b89-afbc-8fd7e0bb8324 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:57.163624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a134e1e-7ed0-4b89-afbc-8fd7e0bb8324 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:57.164377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a134e1e-7ed0-4b89-afbc-8fd7e0bb8324 None admin] RBAC: Adding query filter params (group.id=0edb1a6bf3da4bc5a666884fb73a8c19, role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=ee25af34c3554966b7a70b6f9a37af9a) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:57.164686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a134e1e-7ed0-4b89-afbc-8fd7e0bb8324 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:57.166178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a134e1e-7ed0-4b89-afbc-8fd7e0bb8324 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:57.171790 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 178/335] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Oct 23 00:42:57 2022] GET /identity/v3/role_assignments?group.id=0edb1a6bf3da4bc5a666884fb73a8c19&scope.project.id=ee25af34c3554966b7a70b6f9a37af9a&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:42:57.171908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d248db-f7df-4edd-bc46-5f4c743277d7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:42:57.209122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4f438-5c72-45f5-9337-51fa240831c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:42:57.209742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4f438-5c72-45f5-9337-51fa240831c5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:42:57.210655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4f438-5c72-45f5-9337-51fa240831c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:42:57.210939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4f438-5c72-45f5-9337-51fa240831c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:42:57.211680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4f438-5c72-45f5-9337-51fa240831c5 None admin] RBAC: Adding query filter params (group.id=150041a347474b7f898024889f965044, role.id=633fd4ad499a4f62a5a2cbac0eb861ce, scope.project.id=9abcb12b59954bc4b95706027f575891) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:42:57.211982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4f438-5c72-45f5-9337-51fa240831c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:42:57.213453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f4f438-5c72-45f5-9337-51fa240831c5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:42:57.220334 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 158/336] 10.222.0.39 () {56 vars in 1376 bytes} [Sun Oct 23 00:42:57 2022] GET /identity/v3/role_assignments?group.id=150041a347474b7f898024889f965044&scope.project.id=9abcb12b59954bc4b95706027f575891&role.id=633fd4ad499a4f62a5a2cbac0eb861ce => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:00.096835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f19418-56aa-48b5-8443-930b8e922981 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.097366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f19418-56aa-48b5-8443-930b8e922981 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.097366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f19418-56aa-48b5-8443-930b8e922981 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.098885 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 179/337] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:00.105171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8fdb4-f93d-416c-8c42-15c5415b59a3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.105678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8fdb4-f93d-416c-8c42-15c5415b59a3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.105776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8fdb4-f93d-416c-8c42-15c5415b59a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.133745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e8d8fdb4-f93d-416c-8c42-15c5415b59a3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:00.134999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e8d8fdb4-f93d-416c-8c42-15c5415b59a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:00.181871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc546021-c6f3-40c0-90ac-9c9b20a67ce3 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.182175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc546021-c6f3-40c0-90ac-9c9b20a67ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.182426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc546021-c6f3-40c0-90ac-9c9b20a67ce3 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.183795 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 180/338] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:00.200637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e732c40-a513-46c1-afb8-155d0a28e717 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.201038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e732c40-a513-46c1-afb8-155d0a28e717 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.201321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e732c40-a513-46c1-afb8-155d0a28e717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.233904 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 159/339] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:43:00.234819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9e732c40-a513-46c1-afb8-155d0a28e717 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:00.236022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9e732c40-a513-46c1-afb8-155d0a28e717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:00.314662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b85f-395e-4cb1-88da-40a11dd32521 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.315015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b85f-395e-4cb1-88da-40a11dd32521 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.315243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185b85f-395e-4cb1-88da-40a11dd32521 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.316954 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 160/340] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:00.322912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8199ac-fe96-462a-8b08-062e02bba6e9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.323241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8199ac-fe96-462a-8b08-062e02bba6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.323466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8199ac-fe96-462a-8b08-062e02bba6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.345917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1d8199ac-fe96-462a-8b08-062e02bba6e9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:00.347514 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 181/341] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:00.348965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1d8199ac-fe96-462a-8b08-062e02bba6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:00.431742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:00.449282 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 161/342] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:43:00.557947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e732c40-a513-46c1-afb8-155d0a28e717 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:00.656696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:00.689972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e732c40-a513-46c1-afb8-155d0a28e717 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:00.695406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8199ac-fe96-462a-8b08-062e02bba6e9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:00.747002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:00.747658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.747934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.748205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.760032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:00.761542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:00.788974 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:00.789991 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:00.790871 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:00.790871 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:00.790871 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:00.790871 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 182/343] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:00.796027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:00.867033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8199ac-fe96-462a-8b08-062e02bba6e9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:00.903266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:00.904220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.904529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.904801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.910319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dd08af-02ee-432e-8ef4-3bad984b1308 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:00.914523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:00.915957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:00.930271 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:00.931163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:00.931993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:00.931993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:00.931993 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:00.931993 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 162/344] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:00.932213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:00.943156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:00.952900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72407916-d9d7-4133-b035-e85c908cb0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:00.953816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407916-d9d7-4133-b035-e85c908cb0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.954286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407916-d9d7-4133-b035-e85c908cb0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.954789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72407916-d9d7-4133-b035-e85c908cb0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.955602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72407916-d9d7-4133-b035-e85c908cb0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:00.956079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72407916-d9d7-4133-b035-e85c908cb0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:00.957750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72407916-d9d7-4133-b035-e85c908cb0a8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:00.975537 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 183/345] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:00.977040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924307-fc45-428f-9d36-1a0362faf6df None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:00.977506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924307-fc45-428f-9d36-1a0362faf6df None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:00.977967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5924307-fc45-428f-9d36-1a0362faf6df None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:00.978098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:00.979265 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 184/346] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:00.979943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5924307-fc45-428f-9d36-1a0362faf6df None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:01.040681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314befd8-8a0a-4119-81bd-d1ad75431509 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.077622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.078341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.078625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.078890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.090403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.090935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5924307-fc45-428f-9d36-1a0362faf6df None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.091838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:01.108280 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:01.109643 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:01.110463 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:01.110463 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:01.110463 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:01.110463 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 163/347] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:01.114871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:01.125855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13345099-7150-4547-af25-61492015c4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.126472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13345099-7150-4547-af25-61492015c4e2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.126751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13345099-7150-4547-af25-61492015c4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.127021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13345099-7150-4547-af25-61492015c4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.127830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13345099-7150-4547-af25-61492015c4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:01.127950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13345099-7150-4547-af25-61492015c4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.129369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13345099-7150-4547-af25-61492015c4e2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.145709 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 185/348] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:01.145709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddeea1-e3b4-4484-b747-6f18c19ff9ac None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.145709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddeea1-e3b4-4484-b747-6f18c19ff9ac None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.145709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ddeea1-e3b4-4484-b747-6f18c19ff9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.164987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-81ddeea1-e3b4-4484-b747-6f18c19ff9ac None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:01.166206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-81ddeea1-e3b4-4484-b747-6f18c19ff9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:01.212124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:01.219305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:01.229591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8367474-ec5b-4299-b865-de95359f5e8c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.239223 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 186/349] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:01.240108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddeea1-e3b4-4484-b747-6f18c19ff9ac None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:01.266369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.267630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.268600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.269373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.270223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.272029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.298980 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 164/350] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:43:01 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:01.300221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:01.436151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ddeea1-e3b4-4484-b747-6f18c19ff9ac None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.496332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e3938-f2b1-4c9f-b88b-9f59b6ae18a5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.517200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.517815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.518091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.518360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.518999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:01.519295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.520728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.534013 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 187/351] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:01.534871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:01.551435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.552609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.552892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.553336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.554594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.556106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.582081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:01.585465 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 165/352] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:43:01 2022] POST /identity/v3/users => generated 248 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:01.586196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:01.590910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:01.690531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc3b9e-9dc7-488d-9b57-3bc94462156c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.729851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1477028a-3729-4f8d-bbf3-f452fd77dd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.730089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14acd44-f23c-415a-b853-8970e154685e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.730498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477028a-3729-4f8d-bbf3-f452fd77dd9f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.730775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477028a-3729-4f8d-bbf3-f452fd77dd9f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.731048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1477028a-3729-4f8d-bbf3-f452fd77dd9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.745832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477028a-3729-4f8d-bbf3-f452fd77dd9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.748092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1477028a-3729-4f8d-bbf3-f452fd77dd9f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1477028a-3729-4f8d-bbf3-f452fd77dd9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:01.768412 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:01.769381 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:01.770218 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:01.770218 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:01.770218 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:01.770374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff09d76-906c-4fcb-96f4-8176c6205ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.770374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff09d76-906c-4fcb-96f4-8176c6205ba0 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.770508 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 188/353] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:01.770598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff09d76-906c-4fcb-96f4-8176c6205ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.770598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff09d76-906c-4fcb-96f4-8176c6205ba0 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.770598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff09d76-906c-4fcb-96f4-8176c6205ba0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.770781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eebe1-a8c9-4a72-9612-348f97dd82d5 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.771591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eebe1-a8c9-4a72-9612-348f97dd82d5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.771591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0eebe1-a8c9-4a72-9612-348f97dd82d5 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.772703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff09d76-906c-4fcb-96f4-8176c6205ba0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.777688 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 189/354] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:01.778444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0eebe1-a8c9-4a72-9612-348f97dd82d5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:01.802793 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 166/355] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:43:01 2022] POST /identity/v3/users => generated 248 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:01.804725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1efae-4e03-46e8-858e-2386907ae293 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.805013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1efae-4e03-46e8-858e-2386907ae293 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.805277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e1efae-4e03-46e8-858e-2386907ae293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.835650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c0e1efae-4e03-46e8-858e-2386907ae293 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:01.837020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0e1efae-4e03-46e8-858e-2386907ae293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:01.902829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:01.917392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:01.918109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0eebe1-a8c9-4a72-9612-348f97dd82d5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:01.930626 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 167/356] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:01.976466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:01.977027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:01.977418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:01.977690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:01.978349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:01.978699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:01.980178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:01.999473 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 190/357] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:02.006639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1efae-4e03-46e8-858e-2386907ae293 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:02.111456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e1efae-4e03-46e8-858e-2386907ae293 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:02.147817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:02.148456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:02.148737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:02.149007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:02.149652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:02.151078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:02.175022 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 168/358] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:43:02 2022] POST /identity/v3/users => generated 249 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:02.243941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:02.383840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b123-5b27-4b9a-9b5a-fc614f61ff3d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:02.421904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:02.422533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:02.422811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:02.423084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:02.423767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:02.425219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:02.435911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.sql.core [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 00:43:02.435911 dv-cinder-master-3-smb devstack@keystone.service[81372]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 23 00:43:02.435911 dv-cinder-master-3-smb devstack@keystone.service[81372]: [parameters: {'id': 'bdde066631114024bd19d9383208ef16', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 23 00:43:02.435911 dv-cinder-master-3-smb devstack@keystone.service[81372]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81372) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:02.443870 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return ref.to_dict() Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:02.445861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application yield resource Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise self.value Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application session.commit() Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 00:43:02.447994 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.flush() Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application compat.raise_( Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 00:43:02.450039 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:02.452512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:02.454492 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [parameters: {'id': 'bdde066631114024bd19d9383208ef16', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:02.461760 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 23 00:43:02.463692 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:02.465670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 191/359] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Oct 23 00:43:02 2022] POST /identity/v3/roles => generated 144 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:02.465807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:02.510324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:02.575537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038dbd2-84b7-4da5-81fe-f72fd88b6a81 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:02.610555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4375f9-db2e-453c-8bea-e7f03265df09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:02.611494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4375f9-db2e-453c-8bea-e7f03265df09 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:02.611948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4375f9-db2e-453c-8bea-e7f03265df09 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:02.612454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4375f9-db2e-453c-8bea-e7f03265df09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:02.629533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4375f9-db2e-453c-8bea-e7f03265df09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8d4375f9-db2e-453c-8bea-e7f03265df09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:02.635636 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:02.636548 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 00:43:02.637360 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:02.644208 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 169/360] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:02.645739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:02.662485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:02.787135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a98b1-9c41-4009-949e-ffe26036fcd9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:02.838483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1384954-68d3-4b18-8a5d-340420352040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:02.839101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384954-68d3-4b18-8a5d-340420352040 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:02.839382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384954-68d3-4b18-8a5d-340420352040 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:02.839649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1384954-68d3-4b18-8a5d-340420352040 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:02.840333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1384954-68d3-4b18-8a5d-340420352040 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:02.840637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1384954-68d3-4b18-8a5d-340420352040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:02.842002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1384954-68d3-4b18-8a5d-340420352040 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:02.856512 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 192/361] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:03.860633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab149159-f45a-42e8-b6b8-b8892c21ee8a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:03.861563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab149159-f45a-42e8-b6b8-b8892c21ee8a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:03.862028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab149159-f45a-42e8-b6b8-b8892c21ee8a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:03.863566 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 170/362] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:03.868889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a748f6-0a35-4dc8-8714-a555b14443ab None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:03.869671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a748f6-0a35-4dc8-8714-a555b14443ab None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:03.870169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a748f6-0a35-4dc8-8714-a555b14443ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:03.896536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c3a748f6-0a35-4dc8-8714-a555b14443ab None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:03.898686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3a748f6-0a35-4dc8-8714-a555b14443ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:03.990769 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 193/363] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:04.210687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dea8b-e22b-48a8-bb91-6f8143d7ff6b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.211293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dea8b-e22b-48a8-bb91-6f8143d7ff6b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.211753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dea8b-e22b-48a8-bb91-6f8143d7ff6b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.213526 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 171/364] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:04.218945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a633f4a-124e-49ae-ba34-20825499a265 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.219438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a633f4a-124e-49ae-ba34-20825499a265 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.219936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a633f4a-124e-49ae-ba34-20825499a265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.233042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:04.239682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3a633f4a-124e-49ae-ba34-20825499a265 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:04.241656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3a633f4a-124e-49ae-ba34-20825499a265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:04.332428 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 194/365] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:04.383491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dea8b-e22b-48a8-bb91-6f8143d7ff6b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:04.525971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dea8b-e22b-48a8-bb91-6f8143d7ff6b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:04.538959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:04.573776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:04.574409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.574686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.574959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.593164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:04.597972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:04.598912 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:04.599693 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 172/366] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:04.605323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a633f4a-124e-49ae-ba34-20825499a265 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:04.629585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:04.726551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a633f4a-124e-49ae-ba34-20825499a265 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:04.764152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:04.764942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.765222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.765556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.766420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:04.766727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:04.768101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:04.780709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:04.781806 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 195/367] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:04.783549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:04.848225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a638c9-9cfe-46de-82d6-ce86dabfebe0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:04.883919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f7a2c2-847c-489e-8d0f-918782959b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:04.885712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a2c2-847c-489e-8d0f-918782959b8e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.886207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a2c2-847c-489e-8d0f-918782959b8e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.886675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f7a2c2-847c-489e-8d0f-918782959b8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.887994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ebdac-227b-4e74-847c-ce373ee95678 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:04.898274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f7a2c2-847c-489e-8d0f-918782959b8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-53f7a2c2-847c-489e-8d0f-918782959b8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:04.903333 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:04.904332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:04.905114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:04.907764 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 173/368] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:04.909253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98094f-b7fd-466b-9543-2620513ac838 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.909748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98094f-b7fd-466b-9543-2620513ac838 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.910206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98094f-b7fd-466b-9543-2620513ac838 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.911656 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 174/369] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:04.917168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb1442-776d-4035-9904-9f248a735c8a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.917671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb1442-776d-4035-9904-9f248a735c8a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.918140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb1442-776d-4035-9904-9f248a735c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.923330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:04.923963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:04.924242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:04.924542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:04.936017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:04.937604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:04.939382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4fbb1442-776d-4035-9904-9f248a735c8a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:04.940838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4fbb1442-776d-4035-9904-9f248a735c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:04.952745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:04.953646 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:04.954479 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:04.954479 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:04.954479 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:04.954479 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 196/370] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:04.955124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:04.967421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.010917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.022015 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 175/371] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:05.029624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb1442-776d-4035-9904-9f248a735c8a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.064058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1730e0a-6108-43a5-9947-5f25cb190ba7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.108780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91343461-9d48-40a6-b8fa-3404cade6a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.110285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91343461-9d48-40a6-b8fa-3404cade6a4c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.110794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91343461-9d48-40a6-b8fa-3404cade6a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.111495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91343461-9d48-40a6-b8fa-3404cade6a4c None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.112663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91343461-9d48-40a6-b8fa-3404cade6a4c None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:05.113167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91343461-9d48-40a6-b8fa-3404cade6a4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:05.115341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91343461-9d48-40a6-b8fa-3404cade6a4c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:05.127048 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 197/372] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:05.130098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37429426-cfcd-4ef5-a543-7a87569c92e9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.130387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37429426-cfcd-4ef5-a543-7a87569c92e9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.132372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37429426-cfcd-4ef5-a543-7a87569c92e9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.134068 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 198/373] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:05.135004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37429426-cfcd-4ef5-a543-7a87569c92e9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.166748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb1442-776d-4035-9904-9f248a735c8a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.205670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94976938-0da6-48f1-bf29-882d114a7bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.206285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94976938-0da6-48f1-bf29-882d114a7bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.206564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94976938-0da6-48f1-bf29-882d114a7bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.206883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94976938-0da6-48f1-bf29-882d114a7bd9 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.207530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94976938-0da6-48f1-bf29-882d114a7bd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:05.207879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94976938-0da6-48f1-bf29-882d114a7bd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:05.209607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94976938-0da6-48f1-bf29-882d114a7bd9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:05.222067 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 176/374] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:05.227694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4b013-9285-4565-b12a-e448012528af None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.228054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4b013-9285-4565-b12a-e448012528af None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.228244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b4b013-9285-4565-b12a-e448012528af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.247172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-11b4b013-9285-4565-b12a-e448012528af None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:05.248489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-11b4b013-9285-4565-b12a-e448012528af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:05.262851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.274142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37429426-cfcd-4ef5-a543-7a87569c92e9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.288843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.312542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.313181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.313800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.314137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.328426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:05.330634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:05.338591 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 177/375] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:05.339699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4b013-9285-4565-b12a-e448012528af None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:05.352560 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:05.353512 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:05.354352 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:05.354352 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:05.354352 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:05.354352 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 199/376] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:05.354967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.460729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c26fed-3810-4202-9b74-952bfa1a80f8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.473246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b4b013-9285-4565-b12a-e448012528af None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.498375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.499005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.499277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.499544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.510556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:05.515788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:05.516742 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:05.517550 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:05.521959 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 200/377] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:05.523211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.526254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.527153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.527544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.527881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.535826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:05.537572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:05.551098 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:05.552022 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:05.552938 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 178/378] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:05.553568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.560666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.568967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.645924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1668ce-049f-4274-8b86-05ae0ee2abec None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.672816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02629900-5d97-48dc-bab7-bdd3b1dc596b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.685118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.685725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.686001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.686271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.686902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:05.687202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:05.688666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:05.700784 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 201/379] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:05.701655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.711310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.712047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.712359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.712639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.713344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:05.713635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:05.715075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:05.729904 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 179/380] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:05.729904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.795053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.799738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:05.818171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aade35-f4a9-48e1-bac0-a911b9b00c41 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.841546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520f345-1b81-4562-8766-6be70265e831 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.852323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.852937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.853218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.853489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.854219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:05.854522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:05.855939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:05.874444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:05.875068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:05.875344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:05.875613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:05.886200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:05.891396 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:05.892491 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:05.893294 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:05.894381 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 202/381] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:05.894381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.896603 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 180/382] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:05.897383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:05.994914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b5f6d-bd38-42b6-aadc-9a53991626a7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:05.995106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b12c67-d049-4f63-abda-52c8b33e1e35 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.021064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.028545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e582ba92-66b9-4a06-94ff-323ad42e13c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.029162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582ba92-66b9-4a06-94ff-323ad42e13c6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.029338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.029445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582ba92-66b9-4a06-94ff-323ad42e13c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.029689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e582ba92-66b9-4a06-94ff-323ad42e13c6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.029961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.030242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.030515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.036859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582ba92-66b9-4a06-94ff-323ad42e13c6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.038294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e582ba92-66b9-4a06-94ff-323ad42e13c6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.040898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.042310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-e582ba92-66b9-4a06-94ff-323ad42e13c6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:06.050417 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:06.051282 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 203/383] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68f6c-1316-4d03-a426-e974106525f9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.057963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68f6c-1316-4d03-a426-e974106525f9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.058660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f68f6c-1316-4d03-a426-e974106525f9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.058660 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 204/384] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:06.058660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f68f6c-1316-4d03-a426-e974106525f9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:06.058859 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:06.059656 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:06.060525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:06.060525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.060525 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:06.060525 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 181/385] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:06.060525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.062074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.157527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca402e60-9f65-4597-9f9e-599d54deb0d1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.158856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f68f6c-1316-4d03-a426-e974106525f9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.189833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.190421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.190763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.190969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.191617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:06.191912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.192207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecfe56f-82b2-4d5f-a518-173d12674fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.192856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe56f-82b2-4d5f-a518-173d12674fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.193139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe56f-82b2-4d5f-a518-173d12674fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.193302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.193398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecfe56f-82b2-4d5f-a518-173d12674fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.201044 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 182/386] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:06.201840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.203425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfe56f-82b2-4d5f-a518-173d12674fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.204923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecfe56f-82b2-4d5f-a518-173d12674fb7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-aecfe56f-82b2-4d5f-a518-173d12674fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:06.219668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:06.220744 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:06.221574 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:06.221574 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.221574 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:06.221574 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 205/387] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:06.221950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab1018-f940-4849-9107-c6b2c4cf8ea1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.222225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab1018-f940-4849-9107-c6b2c4cf8ea1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.222546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ab1018-f940-4849-9107-c6b2c4cf8ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.240373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-11ab1018-f940-4849-9107-c6b2c4cf8ea1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:06.242413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-11ab1018-f940-4849-9107-c6b2c4cf8ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:06.250463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.289775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.306853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 206/388] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:06.307651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab1018-f940-4849-9107-c6b2c4cf8ea1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.315373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d31291-7f0c-4b6c-bffb-a322128a0404 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.352750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.353372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.353790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.354058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.355150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:06.355610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.357115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.395959 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 183/389] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:06.396742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.409425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ab1018-f940-4849-9107-c6b2c4cf8ea1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.445006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.445629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.445941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.446244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.446906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:06.447237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.448660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.460421 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 207/390] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:06.465538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.499754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.504046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce095c5-aedc-4605-b476-9b21c830399f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.515015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.541027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.541656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.541929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.542194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.552920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.554482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:06.570797 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:06.571759 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:06.577624 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:06.577624 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.577624 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:06.577806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55d227-2bd3-476e-abfe-afdc1bacdefd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.577878 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 184/391] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:06.577878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.609728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.610356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.610637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.610909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.611538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:06.611830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.613210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.624374 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 208/392] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:06.625137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.675667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162cc622-821a-4fbb-9521-7ca44bde9771 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.707206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.707813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.708086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.708379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.718811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.720233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.721274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea0027c-ffde-4a10-93de-e0cfaf83a567 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.723592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:06.734638 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:06.735623 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:06.736580 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:06.736580 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.736580 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:06.736580 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 185/393] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:06.736580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.739947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.760465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.761087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.761365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.761631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.768539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.769969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:06.781840 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:06.782698 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:06.783582 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 209/394] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:06.784127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.839909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718e04c-4718-4006-b6ea-d72a7d4ed8bc None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.872144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.872768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.873040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.873309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.873940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:06.874236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.875591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.884436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffabe6d-0609-41c0-9504-929ab04461c0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:06.887676 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 186/395] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:06.888288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.919115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:06.919736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:06.920009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:06.920274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:06.931463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:06.932892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:06.946881 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:06.947904 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:06.948790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:06.948790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:06.948790 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:06.948790 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 210/396] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:06.948790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:06.953777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.967019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:06.991351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9862574-4ed5-462d-8085-c37346c15a40 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.027366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.027980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.028256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.028552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.035022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.036473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.046948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced717a9-fa6a-49e5-aa43-66dd30b55663 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:07.048015 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:07.048869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:07.049710 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 187/397] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:07.050152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.079797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.080435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.080714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.080988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.081621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:07.081922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.083296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.095022 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 211/398] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:07.095797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.145197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5c62d-6c46-452c-85ca-3d013609490a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.168629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.172499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.180704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.181308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.181631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.181856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.182583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:07.182887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.184295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.196519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d0b7bc-8222-408c-96ed-046a2aea0147 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.202186 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 188/399] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:07.202845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.229390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.230012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.230286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.230552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.237739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.239182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:07.250788 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:07.251727 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:07.252622 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 212/400] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:07.253264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.305898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab0c92-5203-4ce6-b9db-6d2cb008694d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.338854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.339454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.339725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.339994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.340660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:07.340956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.342301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.351024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5a69f-bcf1-4621-95f5-86be522c5f4b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.353255 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 189/401] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:07.353818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.385735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.386338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.386612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.386881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.387564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:07.387867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.389313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.393019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.397620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.407540 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 213/402] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:07.408557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.454286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b343150d-a5fc-4500-8730-484810b26658 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.487195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.487823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.488100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.488408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.495467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.496939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.504908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae216c2-6f7c-42e5-ad60-8cb25407e745 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:07.508945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:07.509945 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 190/403] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:07.511100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.537932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.538554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.538827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.539092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.548576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.550011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:07.564529 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:07.565437 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:07.566314 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:07.566314 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:07.566314 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:07.566787 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 214/404] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:07.567682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.612556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece48348-472e-47f2-b958-102f0b3b9319 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.614085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.615782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.646487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.647086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.647387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.647657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.648477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:07.648779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.650207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.666310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1843ca-d799-4bb4-b905-bdebb1873d45 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.668478 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 191/405] 10.222.0.39 () {56 vars in 1139 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:07.669248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.699571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.700171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.700484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.700758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.701391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.702836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.723232 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 215/406] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:43:07 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:07.723960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.790424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed764d-a5ba-493a-a811-76f09fc3e22b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.838481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.839167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.839447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.839662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017c78d-2b1e-4093-9e54-39ea7c100d76 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:07.839764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/f04fb9e1d1bd42ac815078b61f6a17f8/roles/d36d481771e04201bdf12458d1795e34` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.843162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.858316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=f04fb9e1d1bd42ac815078b61f6a17f8, role_id=d36d481771e04201bdf12458d1795e34)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.859767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.874671 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 192/407] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:07 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/f04fb9e1d1bd42ac815078b61f6a17f8/roles/d36d481771e04201bdf12458d1795e34 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:07.875959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:07.892136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:07.892814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:07.893314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:07.893584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:07.894303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:07.894605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:07.897486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:07.897645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:07.910828 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 216/408] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:07.911695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.028633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df69f6b-eefe-4d6b-90d4-cda6671dd80c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.063088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd1a25-ff67-42c0-b30e-4b92fa64f170 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.077176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.077176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.077460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.077597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.078210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:08.078739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.079888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.095428 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 193/409] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:08.096905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.106173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.106806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.107081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.107352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.119001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.120494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:08.135976 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:08.136951 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:08.137915 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:08.137915 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:08.137915 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:08.137915 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 217/410] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:08.138571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.157351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:08.208719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:08.240342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d96a9b-5137-460d-8f07-8808a9a2acc0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.245332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34efcd44-b9af-45ad-85ba-617fdcc7fc6e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.282226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.283072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.283487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.283899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.288865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.289626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.289999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.290362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/b5429182e2fa4e2aa485fc084e32aafd/roles/d36d481771e04201bdf12458d1795e34` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.292158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.293907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:08.306542 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:08.307569 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:08.308668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:08.309493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=b5429182e2fa4e2aa485fc084e32aafd, role_id=d36d481771e04201bdf12458d1795e34)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.310833 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 218/411] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:08.311120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.311971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.328632 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 194/412] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:08 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/b5429182e2fa4e2aa485fc084e32aafd/roles/d36d481771e04201bdf12458d1795e34 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:08.330118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.418306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:08.445946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcd36f5-c83b-473d-9bfa-143e128e8735 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.497649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.498548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.498955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.499373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.500169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:08.500675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.502459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.512369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe7a8f-dd65-4e9f-9076-a86bbb5639d1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.528541 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 219/413] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:08.534135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:08.535377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.552958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.552958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.552958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.552958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.553440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:08.553440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.556959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.594289 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 195/414] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:08.601914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.676726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24f34f-fbb8-463e-af35-bb9bdb2cf8f5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.735257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f920c80c-6b1e-4d71-bd04-968bedd7bacb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:08.749564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.750215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.750492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.750766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.754056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:08.760930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.762437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.773238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:08.773848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:08.774166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:08.774439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/9075ccb9e3cd46cc9e77caa4060e3c4e/roles/d36d481771e04201bdf12458d1795e34` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:08.782083 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:08.783097 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:08.784264 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 220/415] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:08.793249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:08.795000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=9075ccb9e3cd46cc9e77caa4060e3c4e, role_id=d36d481771e04201bdf12458d1795e34)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:08.796532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:08.816176 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 196/416] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:08 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/9075ccb9e3cd46cc9e77caa4060e3c4e/roles/d36d481771e04201bdf12458d1795e34 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:08.983065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2780f7af-664a-4371-8f15-ecde43b4af24 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:09.029003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04827aa-6911-40f9-9f34-b53dcd267512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:09.029623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04827aa-6911-40f9-9f34-b53dcd267512 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:09.029899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04827aa-6911-40f9-9f34-b53dcd267512 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:09.030169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04827aa-6911-40f9-9f34-b53dcd267512 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:09.030863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04827aa-6911-40f9-9f34-b53dcd267512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:09.031162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04827aa-6911-40f9-9f34-b53dcd267512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:09.032713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04827aa-6911-40f9-9f34-b53dcd267512 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:09.053540 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 221/417] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:09.059122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:09.108710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:09.184590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a6d7a-edb5-4414-84b4-0898ac56a49f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:09.225377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e91dd0-b5ae-4e8c-8c1c-b353bddecadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:09.226096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e91dd0-b5ae-4e8c-8c1c-b353bddecadb None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:09.226371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e91dd0-b5ae-4e8c-8c1c-b353bddecadb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:09.226637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e91dd0-b5ae-4e8c-8c1c-b353bddecadb None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/bb5f82d3cfc14eacbe4cc40bcd0dfb20/roles/d36d481771e04201bdf12458d1795e34` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:09.247192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e91dd0-b5ae-4e8c-8c1c-b353bddecadb None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=bb5f82d3cfc14eacbe4cc40bcd0dfb20, role_id=d36d481771e04201bdf12458d1795e34)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:09.248654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e91dd0-b5ae-4e8c-8c1c-b353bddecadb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:09.280697 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 197/418] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:09 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/bb5f82d3cfc14eacbe4cc40bcd0dfb20/roles/d36d481771e04201bdf12458d1795e34 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:10.185084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40c1d-7186-40e5-820e-90325f799b8a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:10.185590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40c1d-7186-40e5-820e-90325f799b8a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:10.185590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb40c1d-7186-40e5-820e-90325f799b8a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:10.187237 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 222/419] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:10.193119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b67eb-01c8-47ee-83c6-b620e51d1d65 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:10.193602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b67eb-01c8-47ee-83c6-b620e51d1d65 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:10.193872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4b67eb-01c8-47ee-83c6-b620e51d1d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:10.214507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5b4b67eb-01c8-47ee-83c6-b620e51d1d65 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:10.215718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b4b67eb-01c8-47ee-83c6-b620e51d1d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:10.309288 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 198/420] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:10.366981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b9971-253d-4b16-82bf-282f994a72ce None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:10.367287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b9971-253d-4b16-82bf-282f994a72ce None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:10.367519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b9971-253d-4b16-82bf-282f994a72ce None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:10.369129 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 223/421] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:10.374315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d8c3b-15c7-48bb-953e-59ca6842c86b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:10.374814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d8c3b-15c7-48bb-953e-59ca6842c86b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:10.375278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d8c3b-15c7-48bb-953e-59ca6842c86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:10.401240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-aa1d8c3b-15c7-48bb-953e-59ca6842c86b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:10.402761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aa1d8c3b-15c7-48bb-953e-59ca6842c86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:10.405858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:10.490599 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 199/422] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:10.618209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b9971-253d-4b16-82bf-282f994a72ce None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:10.677785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:10.724521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b9971-253d-4b16-82bf-282f994a72ce None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:10.767171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:10.767724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:10.767997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:10.768263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:10.785128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:10.790571 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:10.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:10.792697 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 224/423] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:10.798166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d8c3b-15c7-48bb-953e-59ca6842c86b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:10.877602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.024397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caaeee2-8747-4684-b986-ed6011581831 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.046590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d8c3b-15c7-48bb-953e-59ca6842c86b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.048253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:11.064933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd79e7d5-dd6b-438a-98d1-a2a89c669b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.065570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79e7d5-dd6b-438a-98d1-a2a89c669b16 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.065846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79e7d5-dd6b-438a-98d1-a2a89c669b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.066114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd79e7d5-dd6b-438a-98d1-a2a89c669b16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.077178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd79e7d5-dd6b-438a-98d1-a2a89c669b16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-dd79e7d5-dd6b-438a-98d1-a2a89c669b16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:11.082362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:11.083232 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:11.084467 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 225/424] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:11.085948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdcb3-dc97-447e-9953-61375ec7b8a6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.089272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdcb3-dc97-447e-9953-61375ec7b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.089272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfdcb3-dc97-447e-9953-61375ec7b8a6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.090386 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 226/425] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:11.093298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfdcb3-dc97-447e-9953-61375ec7b8a6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.095056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26815c6-fe05-410d-b8ba-e88cd0dfcfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.095671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26815c6-fe05-410d-b8ba-e88cd0dfcfd0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.095947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26815c6-fe05-410d-b8ba-e88cd0dfcfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.096220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26815c6-fe05-410d-b8ba-e88cd0dfcfd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.096922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26815c6-fe05-410d-b8ba-e88cd0dfcfd0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:11.097222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26815c6-fe05-410d-b8ba-e88cd0dfcfd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:11.098579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26815c6-fe05-410d-b8ba-e88cd0dfcfd0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:11.112016 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 200/426] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:11.112016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7efd-eb96-44f7-86f2-da1366d444b3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.112016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7efd-eb96-44f7-86f2-da1366d444b3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.112016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7efd-eb96-44f7-86f2-da1366d444b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.116151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:11.128943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-55aa7efd-eb96-44f7-86f2-da1366d444b3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:11.130265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-55aa7efd-eb96-44f7-86f2-da1366d444b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:11.198578 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 201/427] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:11.199567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa7efd-eb96-44f7-86f2-da1366d444b3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.223734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfdcb3-dc97-447e-9953-61375ec7b8a6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.276720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.277380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.277640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.277905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.278561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:11.278955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:11.280444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:11.290332 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 227/428] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:11.296697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.305427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa7efd-eb96-44f7-86f2-da1366d444b3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.347649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:11.351501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.352141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.352463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.352719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.363419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:11.364957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:11.380671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:11.382445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:11.383270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:11.383270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:11.383270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:11.383270 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 202/429] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:11.389253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.402001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:11.495177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41dd016-d699-43d2-bb20-29dfd6c788b8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.530381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.531329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.531798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.532260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.533136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:11.533626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:11.535175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:11.542836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8be07-6328-4a91-80b0-3c38b8e7bf60 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.548214 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 203/430] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:11.553989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.594105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90c2b6c-bb4c-4bc3-bb1e-dbe0a139199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.594725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c2b6c-bb4c-4bc3-bb1e-dbe0a139199b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.595002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c2b6c-bb4c-4bc3-bb1e-dbe0a139199b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.595268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90c2b6c-bb4c-4bc3-bb1e-dbe0a139199b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.604867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c2b6c-bb4c-4bc3-bb1e-dbe0a139199b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:11.605707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:11.606302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90c2b6c-bb4c-4bc3-bb1e-dbe0a139199b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d90c2b6c-bb4c-4bc3-bb1e-dbe0a139199b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:11.622809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:11.623900 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:11.624890 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:11.624890 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:11.624890 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:11.624890 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 228/431] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:11.625177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c6fa1-614a-430b-8587-7a33c8096ef9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.625410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c6fa1-614a-430b-8587-7a33c8096ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.625738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c6fa1-614a-430b-8587-7a33c8096ef9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.626900 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 229/432] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:11.631005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c6fa1-614a-430b-8587-7a33c8096ef9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.669018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531ab3b-b7ee-430a-9ced-5a1572018b58 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.705869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.706881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.707329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.707792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.715537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:11.717299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:11.719674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:11.730152 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:11.731125 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:11.732000 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:11.732523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c6fa1-614a-430b-8587-7a33c8096ef9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.733854 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 204/433] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:11.734611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.772348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09aaf38-2140-48cc-bdae-864a390de0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.772970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aaf38-2140-48cc-bdae-864a390de0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.773253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aaf38-2140-48cc-bdae-864a390de0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.773526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09aaf38-2140-48cc-bdae-864a390de0b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.783763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09aaf38-2140-48cc-bdae-864a390de0b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a09aaf38-2140-48cc-bdae-864a390de0b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:11.788526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:11.789618 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:11.790656 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 230/434] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:11.791312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792cf61-7924-4728-9b91-46046803b283 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.791381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792cf61-7924-4728-9b91-46046803b283 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.791611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c792cf61-7924-4728-9b91-46046803b283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.811459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c792cf61-7924-4728-9b91-46046803b283 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:11.811459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c792cf61-7924-4728-9b91-46046803b283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:11.861550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789e521-26ac-4d84-97f0-30f3835d5740 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:11.863951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:11.884851 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 231/435] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:11.885925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792cf61-7924-4728-9b91-46046803b283 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.908872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:11.909762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:11.910271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:11.910639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:11.911383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:11.911780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:11.913575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:11.935869 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 205/436] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:11.937032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:11.988460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.016069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c792cf61-7924-4728-9b91-46046803b283 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.050984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.051765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.052141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.052524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.054030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:12.054519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.055976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.057723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ca768f-0a60-4ed2-b0b5-40fbe183502c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.079619 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 232/437] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:12.080733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.098928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.099992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.101357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.101812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.102630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:12.103080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.105182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.115391 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 206/438] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:12.116964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.153746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.198984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53030941-000f-4ba4-a626-9f39b853d30f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.234591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8462b6c9-daeb-4adb-8ddb-c7a005fd6b1f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.236777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.237417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.237700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.237973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.245820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.247519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.249297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:12.260952 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:12.261849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:12.262766 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 233/439] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:12.263763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.277363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.277999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.278279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.278548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.289990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.291432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:12.306984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:12.307852 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:12.308745 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:12.308745 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:12.308745 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:12.312629 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 207/440] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:12.313468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.366827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0265285-89fb-4d3d-8960-590b00f2783c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.390550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.400464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.401099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.401379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.401654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.411979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.413504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22ece1c-0a90-44f6-bac8-3b33eac1dd70 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.413629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:12.427106 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:12.427964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:12.429537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:12.429537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:12.429537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:12.429537 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 234/441] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:12.429753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.454863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.455480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.455753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.456020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.465730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:12.470848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:12.471748 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:12.472612 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 208/442] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:12.473280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.484157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.533903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a906fc48-15cb-4b37-bfc7-159e1c810844 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.566930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a228d5-0877-4ca0-a538-a8435d86e3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.567461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a228d5-0877-4ca0-a538-a8435d86e3d9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.567737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a228d5-0877-4ca0-a538-a8435d86e3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.568004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a228d5-0877-4ca0-a538-a8435d86e3d9 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.568746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a228d5-0877-4ca0-a538-a8435d86e3d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:12.569605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a228d5-0877-4ca0-a538-a8435d86e3d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.569783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11766c16-026e-4389-80a3-cc4108ba758e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.571061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a228d5-0877-4ca0-a538-a8435d86e3d9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.587550 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 235/443] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:12.588774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6d747-7dcc-4724-a047-b082ffb01b9f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.589056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6d747-7dcc-4724-a047-b082ffb01b9f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.589325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6d747-7dcc-4724-a047-b082ffb01b9f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.590681 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 236/444] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:12.591189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6d747-7dcc-4724-a047-b082ffb01b9f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.606049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.606646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.606920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.607189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.607848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:12.608143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.609534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.617694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.623105 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 209/445] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:12.623793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.695935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6d747-7dcc-4724-a047-b082ffb01b9f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.705338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.721589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972596eb-4da2-4b3a-8f2a-a82222b436a3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.729504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.730104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.730381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.730652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.731287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:12.731583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.732978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.744201 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 237/446] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:12.745007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.759576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53208c0-1588-4cef-8377-bad2445123e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.760183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53208c0-1588-4cef-8377-bad2445123e0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.760486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53208c0-1588-4cef-8377-bad2445123e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.760752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53208c0-1588-4cef-8377-bad2445123e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.761400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53208c0-1588-4cef-8377-bad2445123e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:12.761693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53208c0-1588-4cef-8377-bad2445123e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.763029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53208c0-1588-4cef-8377-bad2445123e0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.770590 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 210/447] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:12.771751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3c91-9646-4d3b-b502-4fe75441f3c8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.772032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3c91-9646-4d3b-b502-4fe75441f3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.772319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b3c91-9646-4d3b-b502-4fe75441f3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.789092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1a8b3c91-9646-4d3b-b502-4fe75441f3c8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:12.790247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1a8b3c91-9646-4d3b-b502-4fe75441f3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:12.844147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.849617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5572eb-a0f6-44d8-91c7-098314bcdb00 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:12.855920 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 211/448] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:12.863463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3c91-9646-4d3b-b502-4fe75441f3c8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:12.884229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:12.884858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:12.885133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:12.885816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:12.924109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:12.958448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:12.959899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:12.963886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b3c91-9646-4d3b-b502-4fe75441f3c8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.001538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.002161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.002752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.003026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.010780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.012227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:13.023171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:13.024031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 212/449] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:13.029597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.071623 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 238/450] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:43:13.071623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.082603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.134265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da5b23-2f1a-4d7e-975d-773445e803bd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.139173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.167331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.167954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.168234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.168533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.171928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8207a3a2-ba17-4af0-96c3-a2e977f8e7e7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.175832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.177275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:13.188640 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:13.189561 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 213/451] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:13.190538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.208400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.209029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.209307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.210111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.219800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.221277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:13.235561 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:13.236670 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:13.238230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:13.238230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:13.238230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:13.238230 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 239/452] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:13.238230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.294986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d35d7f-f092-4f2c-b5ac-6cb47a15f2c5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.305435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.330040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.330944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.331228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.331503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.332186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:13.332585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.334061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.337068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df755b3-3285-49f6-8e13-6f3bad50d2fc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.345871 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 214/453] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:13.346785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.357390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.371994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.372643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.372919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.373187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.384364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.385823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:13.399191 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:13.400075 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:13.401173 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:13.401173 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:13.401173 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:13.401173 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 240/454] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:13.401173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.450303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdf48f-faac-496a-bf54-a7202ca35ed5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.485223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.485843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.486118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.486389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.496804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:13.497446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d408c3b-7709-47cf-b05d-1f726853252d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:13.501521 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:13.502565 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 215/455] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:13.503361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.530967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.531572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.531851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.532124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.532903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:13.533207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.534308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.534909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.551222 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 241/456] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:13.551989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.581104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.607296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751872e-4578-49a5-80a1-13250d810229 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.640228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.640962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.641246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.641520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.642160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:13.642470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.643857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.648777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fc6226-9f55-4667-a56c-bf5e3452ddfe None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.654693 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 216/457] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:13.655470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.686249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.686901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.687184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.687452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/0800a1f4140a4d098eadb94eeba922be/roles/d36d481771e04201bdf12458d1795e34` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.701279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=0800a1f4140a4d098eadb94eeba922be, role_id=d36d481771e04201bdf12458d1795e34)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.702703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.714599 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 242/458] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:13 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/0800a1f4140a4d098eadb94eeba922be/roles/d36d481771e04201bdf12458d1795e34 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:13.715375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.759091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.771267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a31c7-ec68-43b3-9154-fed397340fbd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.811035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.811667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.811949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.812219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.812920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:13.813222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.814613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.829949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:13.837216 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 217/459] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:13.837216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.857882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be01b6-e5a6-4725-a0d0-d3fbd710f19c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.899835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.900609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.900889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.901566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:13.901820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:13.902117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:13.903464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:13.911878 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 243/460] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:13.916503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:13.947041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaf15b0-7cc4-4309-ad4f-e4c655332e5d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:13.985517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:13.986121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:13.986893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:13.987170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.008038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.021794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99003c-acec-4f29-901f-19a94981878e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.057028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.057770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.058047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.058320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.062666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.064165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.066435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.067856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:14.079077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:14.079981 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 244/461] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:14.081362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.087483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.172184 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 218/462] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:43:14.173372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.194804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c658c-0526-4067-9fec-abe0411b6772 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.226185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.232274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.233508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.234222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.234643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.242679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.244666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:14.255810 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:14.257010 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:14.257861 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 245/463] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:14.258322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.286163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bbd544-84a1-45ac-850e-0a2bc6232e2d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.323990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.324687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.324898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.325168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.327121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.337880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.338863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:14.355091 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:14.356021 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:14.356903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:14.356903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:14.356903 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:14.356903 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 219/464] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:14.357112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.372715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d8cc12-ce1a-4851-8c24-ea5bc4a2f821 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.407071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.407674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.407949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.408214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.408977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:14.409274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.410690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.429060 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 246/465] 10.222.0.39 () {56 vars in 1137 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:14.429643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.466154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bb3904-da15-4368-86bc-6816ee1f331b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.468003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.503307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.504030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.504410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.504760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.515796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.517307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:14.532701 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:14.533744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:14.534744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:14.534744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:14.534744 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:14.536670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13bc02-e7b3-47ca-9c83-e71e591b7285 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.538410 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 220/466] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:14.539258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.558394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.574855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.575827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.576788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.576978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.577646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:14.577952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.579388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.595991 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 247/467] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:14.597953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.649323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e7a13f-d814-4c92-a02e-5a739f476ef3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.688357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.688994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.689269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.689825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.690150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:14.690472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.691831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.703260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d220cc-1fdc-4ea8-81f7-847b02a1b22b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.703865 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 221/468] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:14.709009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.724557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.738442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.739044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.739316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.739580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.793362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.816619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.817810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.820101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd587999-e472-43ee-a810-6b15a95a9a68 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:14.854484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:14.855094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:14.855374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:14.855653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:14.856317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:14.856621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:14.857985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:14.874099 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 222/469] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:14.874099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.928917 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 248/470] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:43:14.929845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:14.959130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:14.986106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758e385f-0876-4cc8-8801-e4e43ec1f7cd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.014725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:15.025610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.026600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.026936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.027209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.027937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] RBAC: Adding query filter params (role.id=d36d481771e04201bdf12458d1795e34, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=f04fb9e1d1bd42ac815078b61f6a17f8) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:15.028242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.029689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.035360 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 223/471] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/role_assignments?user.id=f04fb9e1d1bd42ac815078b61f6a17f8&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=d36d481771e04201bdf12458d1795e34 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:15.035928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7439db3-233f-4c05-a776-f517718444d9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.036116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.077875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.078523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.078798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.079100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.091996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.093864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:15.115487 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:15.117064 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 00:43:15.117938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:15.120473 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 249/472] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:15.121639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.184037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d12b1-d0e4-44be-b1dd-4fbb810d729f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.228881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.229834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.230488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.230959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.243690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.245586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.247659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:15.252991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979b3e12-0a69-4a7e-9869-4451ac9a0028 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:15.261697 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:15.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:15.263671 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 224/473] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:15.264225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.301575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.302602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.303907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.304501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.318386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.320409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:15.321195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:15.341713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:15.342864 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:15.343897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:15.343897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:15.343897 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:15.345943 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 250/474] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:15.346872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.381947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1c0dd-579e-48c3-a291-3b212db1207d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.419369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.420445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.420833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.421220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.422291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:15.422659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.424196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.444157 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 225/475] 10.222.0.39 () {56 vars in 1139 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:15.449171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.476101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb85e-228e-4210-9a05-43483f22294a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.518848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:15.524854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.525464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.525739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.526005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.526690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:15.526988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.528436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.543455 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 251/476] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:15.544475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.576851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be018255-6923-44c6-aba9-abf35c54cfa7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.609657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:15.621824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.622680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.623126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.623593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.624454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:15.624944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.627261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.641072 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 226/477] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:15.642224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.681103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda773af-078b-40a3-b403-53e77144f14b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.719590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd55d24-2650-423f-83dd-f621651b69da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.720837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd55d24-2650-423f-83dd-f621651b69da None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.721358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd55d24-2650-423f-83dd-f621651b69da None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.721816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd55d24-2650-423f-83dd-f621651b69da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.765745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb38da-b0b1-4df1-b759-42ed7ada43d6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:15.784667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:15.807499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:15.808475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.808954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.809465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.810382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] RBAC: Adding query filter params (role.id=d36d481771e04201bdf12458d1795e34, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=b5429182e2fa4e2aa485fc084e32aafd) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:15.810937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.812644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.818015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd55d24-2650-423f-83dd-f621651b69da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:15.820037 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 227/478] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/role_assignments?user.id=b5429182e2fa4e2aa485fc084e32aafd&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=d36d481771e04201bdf12458d1795e34 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:15.820699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.821138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd55d24-2650-423f-83dd-f621651b69da None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:15.897060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:15.957773 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 252/479] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:43:15.959186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c060c9-60a4-4a15-974d-8c1e1d787647 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:15.959526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c060c9-60a4-4a15-974d-8c1e1d787647 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:15.959860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c060c9-60a4-4a15-974d-8c1e1d787647 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:15.961190 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 253/480] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:15.966707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c060c9-60a4-4a15-974d-8c1e1d787647 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:15.976431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22a696-073d-4db0-83e0-970a827c0320 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.024680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e203184-9ef1-4cff-9f2f-a0d9ef66d322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.025598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203184-9ef1-4cff-9f2f-a0d9ef66d322 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.025883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203184-9ef1-4cff-9f2f-a0d9ef66d322 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.026160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e203184-9ef1-4cff-9f2f-a0d9ef66d322 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.035860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e203184-9ef1-4cff-9f2f-a0d9ef66d322 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:16.038423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e203184-9ef1-4cff-9f2f-a0d9ef66d322 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4e203184-9ef1-4cff-9f2f-a0d9ef66d322 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:16.051497 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:16.052788 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:16.054412 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 228/481] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:16.054958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fa32a-5679-4cc5-8c0f-888e92258ad3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.055120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fa32a-5679-4cc5-8c0f-888e92258ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.055390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fa32a-5679-4cc5-8c0f-888e92258ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.080507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f44fa32a-5679-4cc5-8c0f-888e92258ad3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:16.081828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f44fa32a-5679-4cc5-8c0f-888e92258ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:16.091070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c060c9-60a4-4a15-974d-8c1e1d787647 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.128175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:16.138277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.138920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.139199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.139470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.151836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:16.153433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:16.167218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 229/482] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:16.168268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fa32a-5679-4cc5-8c0f-888e92258ad3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:16.179613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:16.180545 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:16.181442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:16.181442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:16.181442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:16.181442 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 254/483] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:16.189588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:16.193637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:16.336504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fa32a-5679-4cc5-8c0f-888e92258ad3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.376144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.377032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.377602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.379083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.380847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:16.382249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:16.384341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:16.388124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f6777-fb75-4231-9dbf-5639a3fdb8cf None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.407951 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 230/484] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:16.415006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:16.452799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.453805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.454283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.454792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.455630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:16.456135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:16.457734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:16.472075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:16.473938 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 255/485] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:16.485566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:16.567551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe50fd8-df6a-4b31-8bde-4cec3695cdce None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.571568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:16.606233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.606909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.607192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.607463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.608186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] RBAC: Adding query filter params (role.id=d36d481771e04201bdf12458d1795e34, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=9075ccb9e3cd46cc9e77caa4060e3c4e) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:16.608531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:16.610111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:16.624328 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 231/486] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/role_assignments?user.id=9075ccb9e3cd46cc9e77caa4060e3c4e&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=d36d481771e04201bdf12458d1795e34 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:16.624328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:16.630464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f25a06-cae7-4409-b044-6030ff1f4c97 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.707836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.708513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.708793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.709061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.723754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:16.725305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:16.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:16.751950 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:16.752805 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:16.753400 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 256/487] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:16.761982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:16.824727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:16.840980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bb2e6f-ebb5-42b1-a709-53c36e7987bb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.892556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.893118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.894031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.894031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.910355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:16.917410 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:16.918305 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:16.919470 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 232/488] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:16.924956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:16.931649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb4aaf-b39f-40a1-949f-2dc6aeea2fd3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:16.957183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:16.969115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:16.969722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:16.969997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:16.970262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:16.970953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:16.971249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:16.972718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:16.987074 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 257/489] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:16.993336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:17.057498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf145c9-0629-40aa-a5c2-4a78c1ce9711 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:17.097226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:17.097841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.104341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.104341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.104341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:17.104341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:17.104341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:17.110178 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 233/490] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:17.116920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:17.137381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:17.166486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed9dec9-c253-44f4-9cc0-0184d0721e6f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:17.230684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:17.231304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.231952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.232187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.233014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] RBAC: Adding query filter params (role.id=d36d481771e04201bdf12458d1795e34, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=bb5f82d3cfc14eacbe4cc40bcd0dfb20) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:17.233414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:17.234774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:17.235450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af49bb-671b-40cd-92b8-d4d9d6d19eef None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:17.247439 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 258/491] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:16 2022] GET /identity/v3/role_assignments?user.id=bb5f82d3cfc14eacbe4cc40bcd0dfb20&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=d36d481771e04201bdf12458d1795e34 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:17.276037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:17.276973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.277398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.277840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.290061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:17.291778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:17.310940 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:17.311882 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:17.313254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:17.313254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:17.313254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:17.315320 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 234/492] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:17.322505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:17.394041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:17.473505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e12bf-5376-4e10-925c-62491eb32e34 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:17.519368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6924f-3651-466c-b78f-9c474f19b9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:17.519987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6924f-3651-466c-b78f-9c474f19b9b5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.520266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6924f-3651-466c-b78f-9c474f19b9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.520572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6924f-3651-466c-b78f-9c474f19b9b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.521235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6924f-3651-466c-b78f-9c474f19b9b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:17.521539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6924f-3651-466c-b78f-9c474f19b9b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:17.523048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6924f-3651-466c-b78f-9c474f19b9b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:17.536509 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 259/493] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:17.542457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:17.554099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1bdd2-879a-4a1b-a758-fc39b23a24c2 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.554399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1bdd2-879a-4a1b-a758-fc39b23a24c2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.554642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b1bdd2-879a-4a1b-a758-fc39b23a24c2 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.556291 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 260/494] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:17.561400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343b41-f01c-4ba4-9aac-158a7c4e5280 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.561686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343b41-f01c-4ba4-9aac-158a7c4e5280 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.561967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca343b41-f01c-4ba4-9aac-158a7c4e5280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.581064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ca343b41-f01c-4ba4-9aac-158a7c4e5280 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:17.582214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ca343b41-f01c-4ba4-9aac-158a7c4e5280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:17.628389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:17.665138 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 261/495] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 00:43:17.672444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb992ad-a0c8-430c-9eae-b64b40805559 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:17.707171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:17.708161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.708661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.709129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.717900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:17.719574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:17.731953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:17.732916 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:17.733799 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:17.736401 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 235/496] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:17.742297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca343b41-f01c-4ba4-9aac-158a7c4e5280 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:17.792960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:17.871401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca343b41-f01c-4ba4-9aac-158a7c4e5280 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:17.919637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:17.920645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:17.921121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:17.921592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:17.922512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:17.923004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:17.924657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:17.952454 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 262/497] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Oct 23 00:43:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:17.957766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:17.981477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:18.146153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38e0f9b-96f8-4907-9b82-e7922e2a2d26 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:18.172664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:18.180252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c458c6-96df-42b3-93f5-1ee3eb67a3a9 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:18.194335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:18.213223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabd08f-c939-4cd4-9b02-bc67424d339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:18.213836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd08f-c939-4cd4-9b02-bc67424d339a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.214114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd08f-c939-4cd4-9b02-bc67424d339a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.214383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabd08f-c939-4cd4-9b02-bc67424d339a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.219286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:18.219921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.220198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.220495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.232727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:18.237537 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:18.238425 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:18.239325 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:18.239942 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 263/498] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:18.245061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:18.372818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd08f-c939-4cd4-9b02-bc67424d339a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:18.373945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabd08f-c939-4cd4-9b02-bc67424d339a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:18.403998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10291173-99c8-4d06-902e-57d79059aa88 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:18.456401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a91b9-712d-4cee-9e0b-c2b87b26a01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:18.457078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a91b9-712d-4cee-9e0b-c2b87b26a01c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.457360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a91b9-712d-4cee-9e0b-c2b87b26a01c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.457633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a91b9-712d-4cee-9e0b-c2b87b26a01c None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.458307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a91b9-712d-4cee-9e0b-c2b87b26a01c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:18.458607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a91b9-712d-4cee-9e0b-c2b87b26a01c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:18.460574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a91b9-712d-4cee-9e0b-c2b87b26a01c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:18.472421 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 264/499] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:18.473825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73621790-d7a4-4f82-aff1-690db678fde4 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.474120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73621790-d7a4-4f82-aff1-690db678fde4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.474387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73621790-d7a4-4f82-aff1-690db678fde4 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.480895 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 265/500] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:18.481659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73621790-d7a4-4f82-aff1-690db678fde4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:18.507390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:18.518891 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 236/501] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 00:43:18.520324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23304-ffd3-4df2-92c0-644853645fea None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.520792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23304-ffd3-4df2-92c0-644853645fea None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.521067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d23304-ffd3-4df2-92c0-644853645fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.539825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-49d23304-ffd3-4df2-92c0-644853645fea None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:18.541103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-49d23304-ffd3-4df2-92c0-644853645fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:18.564630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:18.609522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73621790-d7a4-4f82-aff1-690db678fde4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:18.614294 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 237/502] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:18.615136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d23304-ffd3-4df2-92c0-644853645fea None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:18.671037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:18.671640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.671923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.672246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.687907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:18.690176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:18.716889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:18.717938 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:18.718809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:18.718809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:18.718809 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:18.719222 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 266/503] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:18.727254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:18.776128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d23304-ffd3-4df2-92c0-644853645fea None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:18.816280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:18.817333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.817830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.818332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.832411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:18.834076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:18.850851 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:18.851793 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:18.852773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:18.852773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:18.852773 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:18.855071 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 238/504] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:18.860005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:18.883464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:18.892522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:18.930733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e7d0a3-ae95-41db-956f-84a7abbb9c05 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:18.995118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:18.996135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:18.996557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:18.997487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:18.998504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:18.998901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.001543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.017987 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 267/505] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:19.018926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.036987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69feaa35-7ddb-4dec-88de-21468f772491 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.072327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.072956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.073235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.073518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.074156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:19.074553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.075956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.089444 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 239/506] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:19.090269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.159271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fcb7f9-8cc1-46fd-8863-1bf3879253d4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.167051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:19.200890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dacd936-82a8-40d7-90f9-e580e8602b86 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.202375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:19.210262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.211538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.211912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.212280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.227240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.228730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.243397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:19.243767 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:19.244861 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:19.245857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:19.245857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:19.245857 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:19.246032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.246032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.252409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.252530 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 268/507] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:19.252530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.255667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.257605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:19.272783 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:19.273737 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:19.274683 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 240/508] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:19.275808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.359899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf6840-9fb0-42b9-a551-5d7a9a8dbc85 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.382863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89d5d9-1300-4cb5-adcf-846459a5491c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.393881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f44cac-0699-4de7-a6de-e0d495d03072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.394515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f44cac-0699-4de7-a6de-e0d495d03072 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.394793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f44cac-0699-4de7-a6de-e0d495d03072 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.395065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f44cac-0699-4de7-a6de-e0d495d03072 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.401696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f44cac-0699-4de7-a6de-e0d495d03072 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.403130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f44cac-0699-4de7-a6de-e0d495d03072 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.412621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-80f44cac-0699-4de7-a6de-e0d495d03072 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:19.414043 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:19.415230 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:19.416103 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 269/509] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:19.416983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae91f4-2b29-416b-aab6-0995d22287fa None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.416983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae91f4-2b29-416b-aab6-0995d22287fa None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.417105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ae91f4-2b29-416b-aab6-0995d22287fa None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.418553 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 270/510] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:19.423082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae91f4-2b29-416b-aab6-0995d22287fa None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.425339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.426239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.426758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.427028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.427669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:19.427968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.429985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.448330 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 241/511] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:19.448330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.470957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:19.529476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ae91f4-2b29-416b-aab6-0995d22287fa None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.561905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250baae-a762-4298-90b5-fc48823f7309 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.564748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd2d42-a3ea-4c8b-b638-b1f3199fdfb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.565363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd2d42-a3ea-4c8b-b638-b1f3199fdfb3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.565643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd2d42-a3ea-4c8b-b638-b1f3199fdfb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.565917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd2d42-a3ea-4c8b-b638-b1f3199fdfb3 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.566710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd2d42-a3ea-4c8b-b638-b1f3199fdfb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:19.567046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd2d42-a3ea-4c8b-b638-b1f3199fdfb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.568503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedd2d42-a3ea-4c8b-b638-b1f3199fdfb3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.591629 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 271/512] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:19.591629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c1da0-bbd2-41e2-af8a-6d4b193de686 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.591629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c1da0-bbd2-41e2-af8a-6d4b193de686 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.591629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c1da0-bbd2-41e2-af8a-6d4b193de686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.604520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.605132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.605394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.605664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.606353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:19.606649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.607432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f20c1da0-bbd2-41e2-af8a-6d4b193de686 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:19.608049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.608691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f20c1da0-bbd2-41e2-af8a-6d4b193de686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:19.626773 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 242/513] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:19.626773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.679054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:19.692323 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 272/514] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:19.692323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c1da0-bbd2-41e2-af8a-6d4b193de686 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.723951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:19.744787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a892e32-1b37-4ab7-9d01-fb6c31bba7c6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.785567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.786285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.786572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.786844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.788034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.789621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.818941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c1da0-bbd2-41e2-af8a-6d4b193de686 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:19.824534 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 243/515] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:43:19 2022] POST /identity/v3/users => generated 254 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:19.833392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:19.887343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888e6fe-777e-4511-9cf1-00d7a8631f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:19.888001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888e6fe-777e-4511-9cf1-00d7a8631f9c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.888275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888e6fe-777e-4511-9cf1-00d7a8631f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.888563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888e6fe-777e-4511-9cf1-00d7a8631f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.914732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888e6fe-777e-4511-9cf1-00d7a8631f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:19.916223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888e6fe-777e-4511-9cf1-00d7a8631f9c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-8888e6fe-777e-4511-9cf1-00d7a8631f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:19.938655 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:19.939683 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:19.940611 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:19.940611 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:19.940611 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:19.945277 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 273/516] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:19.946949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f99f9-e14f-4c56-a69a-bbb3f0b62871 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:19.947245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f99f9-e14f-4c56-a69a-bbb3f0b62871 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:19.947490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76f99f9-e14f-4c56-a69a-bbb3f0b62871 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:19.949165 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 274/517] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:19.949877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f99f9-e14f-4c56-a69a-bbb3f0b62871 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.011048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d83bed-2cb8-4567-bbbe-8f411a67b4e7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.019267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.048351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.056377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4713bd2-fd49-4522-be2f-684b4cab1ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.056967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713bd2-fd49-4522-be2f-684b4cab1ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.057242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713bd2-fd49-4522-be2f-684b4cab1ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.057513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4713bd2-fd49-4522-be2f-684b4cab1ea4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.058291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713bd2-fd49-4522-be2f-684b4cab1ea4 None admin] RBAC: Adding query filter params (role.id=d36d481771e04201bdf12458d1795e34, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=0800a1f4140a4d098eadb94eeba922be) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:20.058643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713bd2-fd49-4522-be2f-684b4cab1ea4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.060261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4713bd2-fd49-4522-be2f-684b4cab1ea4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.066398 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 244/518] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/role_assignments?user.id=0800a1f4140a4d098eadb94eeba922be&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=d36d481771e04201bdf12458d1795e34 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:20.067598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4f3d-2252-4525-ae93-ccce808021ab None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.067882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4f3d-2252-4525-ae93-ccce808021ab None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.068258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4f3d-2252-4525-ae93-ccce808021ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.071284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76f99f9-e14f-4c56-a69a-bbb3f0b62871 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.087915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c6bf4f3d-2252-4525-ae93-ccce808021ab None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:20.089340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c6bf4f3d-2252-4525-ae93-ccce808021ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:20.117943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.118743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.119182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.119467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.120589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:20.121181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.122869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.139248 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 275/519] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:20.140227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.172471 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 245/520] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:20.173437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4f3d-2252-4525-ae93-ccce808021ab None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.289838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.292712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4f3d-2252-4525-ae93-ccce808021ab None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.299932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9009b71-ef75-4192-b0d4-791e7430c686 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.329831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.330467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.330747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.331019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.337983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.338561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.338671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.338872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.339138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.339716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.340008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.341246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:20.351373 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:20.352264 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:20.353593 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 246/521] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:20.354071 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:20.355165 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:20.356277 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 276/522] 10.222.0.39 () {56 vars in 1129 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:20.360463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.361668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.384686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.485213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0a839-8047-4c1a-9c4c-ee65e1f09349 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.504199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db64abbf-ebfb-474f-ab43-83dad912765b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.525641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.527049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.527746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.528190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.529164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:20.529682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.531262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.537162 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 247/523] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:20.538364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.540237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.540945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.541282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.541707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.542465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:20.542885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.544437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.556469 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 277/524] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:20.557388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.570039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.625372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.671948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee081e-eecc-43b8-9845-ed10c6044f8a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.673625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba04303-06b6-46b1-9e66-3b74ef787638 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.708581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.709893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.710230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.710782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] PATH_INFO: `/v3/services/network` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.711239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.712758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.719928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.720807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.721255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.721691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.722493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:20.722964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.724585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:20.724694 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:20.726211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:20.727018 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:20.731702 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 248/525] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:20.732380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.735459 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 278/526] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:20.737459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:20.848583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.868687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f9f90-119d-4bf0-bfdf-4d86e41efd5f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.875581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da1188-70c8-4e42-a39a-427999e0fed1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:20.906973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:20.917245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.918230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.918605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.919096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/f04fb9e1d1bd42ac815078b61f6a17f8/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.922113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=f04fb9e1d1bd42ac815078b61f6a17f8, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.923848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.926616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f22f7-5cad-4400-8e8a-791e6c3861e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:20.927435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f22f7-5cad-4400-8e8a-791e6c3861e2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:20.927890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f22f7-5cad-4400-8e8a-791e6c3861e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:20.928431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f22f7-5cad-4400-8e8a-791e6c3861e2 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:20.929268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f22f7-5cad-4400-8e8a-791e6c3861e2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:20.929598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f22f7-5cad-4400-8e8a-791e6c3861e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:20.931115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f22f7-5cad-4400-8e8a-791e6c3861e2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:20.939205 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 249/527] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:20 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/f04fb9e1d1bd42ac815078b61f6a17f8/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:20.942432 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 279/528] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:20.948276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:21.077053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7994d-2e9a-4fb5-b2d3-4617a74a9008 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:21.122566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e53988-6902-49ff-ae92-c9d72a48e40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:21.123106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e53988-6902-49ff-ae92-c9d72a48e40d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:21.123377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e53988-6902-49ff-ae92-c9d72a48e40d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:21.123646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e53988-6902-49ff-ae92-c9d72a48e40d None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:21.124276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e53988-6902-49ff-ae92-c9d72a48e40d None admin] RBAC: Adding query filter params (type=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:21.124601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e53988-6902-49ff-ae92-c9d72a48e40d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:21.126102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e53988-6902-49ff-ae92-c9d72a48e40d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:21.132937 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 250/529] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:20 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:22.586362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e795c4-359b-42e6-bca8-d6f566e4867c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:22.587373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e795c4-359b-42e6-bca8-d6f566e4867c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:22.587853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e795c4-359b-42e6-bca8-d6f566e4867c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:22.589654 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 280/530] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:22.594543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7012c-7a58-4772-8e0b-028233905d43 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:22.594989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7012c-7a58-4772-8e0b-028233905d43 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:22.595194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7012c-7a58-4772-8e0b-028233905d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:22.621594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6cb7012c-7a58-4772-8e0b-028233905d43 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:22.622936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6cb7012c-7a58-4772-8e0b-028233905d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:22.632722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:22.711244 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 251/531] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:22.749345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1c1d-3085-4ffb-9501-dd5456b42bda None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:22.749650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1c1d-3085-4ffb-9501-dd5456b42bda None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:22.749901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eed1c1d-3085-4ffb-9501-dd5456b42bda None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:22.751527 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 281/532] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:22.757582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bca32-d50d-458b-bfa4-723dafee41d9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:22.757868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bca32-d50d-458b-bfa4-723dafee41d9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:22.758144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867bca32-d50d-458b-bfa4-723dafee41d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:22.785259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-867bca32-d50d-458b-bfa4-723dafee41d9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:22.786426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-867bca32-d50d-458b-bfa4-723dafee41d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:22.877736 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 252/533] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:23.005547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed1c1d-3085-4ffb-9501-dd5456b42bda None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:23.141991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eed1c1d-3085-4ffb-9501-dd5456b42bda None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:23.189005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:23.189648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.189932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.190205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.203129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:23.205241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bca32-d50d-458b-bfa4-723dafee41d9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:23.208056 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:23.208992 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:23.209853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 282/534] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:23.215171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:23.305364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:23.332080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:23.336323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867bca32-d50d-458b-bfa4-723dafee41d9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:23.373125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:23.373779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.374063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.374338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.386608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:23.391544 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:23.392554 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:23.393390 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 253/535] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:23.399014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:23.482219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c0398d-1892-4ee0-97cd-d173315a0663 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:23.534787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73f7d-c2d9-40e8-8e77-733ca36407f9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:23.538533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12865fc-af4f-44ab-892e-3c808a32fa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:23.539211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12865fc-af4f-44ab-892e-3c808a32fa29 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.539493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12865fc-af4f-44ab-892e-3c808a32fa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.539772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12865fc-af4f-44ab-892e-3c808a32fa29 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.540475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12865fc-af4f-44ab-892e-3c808a32fa29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:23.540775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12865fc-af4f-44ab-892e-3c808a32fa29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:23.542147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12865fc-af4f-44ab-892e-3c808a32fa29 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:23.559433 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 283/536] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:23.562918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaff11-2d29-43c3-ac84-3d2b7110575e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.563204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaff11-2d29-43c3-ac84-3d2b7110575e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.563475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaff11-2d29-43c3-ac84-3d2b7110575e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.564757 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 284/537] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:23.565611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaff11-2d29-43c3-ac84-3d2b7110575e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:23.575587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:23.581838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e225c9d-a74a-4d38-acf0-f25900433a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:23.582844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225c9d-a74a-4d38-acf0-f25900433a5a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.583565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225c9d-a74a-4d38-acf0-f25900433a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.584013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e225c9d-a74a-4d38-acf0-f25900433a5a None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.584839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e225c9d-a74a-4d38-acf0-f25900433a5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:23.585566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e225c9d-a74a-4d38-acf0-f25900433a5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:23.587271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e225c9d-a74a-4d38-acf0-f25900433a5a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:23.597973 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 254/538] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:23.599548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df1ee1-2d2d-4e85-b0cb-c1cc0b10bc44 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.600479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df1ee1-2d2d-4e85-b0cb-c1cc0b10bc44 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.600846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df1ee1-2d2d-4e85-b0cb-c1cc0b10bc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.623483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-02df1ee1-2d2d-4e85-b0cb-c1cc0b10bc44 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:23.625162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02df1ee1-2d2d-4e85-b0cb-c1cc0b10bc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:23.695080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaff11-2d29-43c3-ac84-3d2b7110575e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:23.711388 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 255/539] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:23.713147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df1ee1-2d2d-4e85-b0cb-c1cc0b10bc44 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:23.730227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:23.764144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dff41c-6339-44b0-9731-d0735ed4be67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:23.764837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dff41c-6339-44b0-9731-d0735ed4be67 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.765117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dff41c-6339-44b0-9731-d0735ed4be67 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.765389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dff41c-6339-44b0-9731-d0735ed4be67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.790189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dff41c-6339-44b0-9731-d0735ed4be67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:23.791680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dff41c-6339-44b0-9731-d0735ed4be67 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-80dff41c-6339-44b0-9731-d0735ed4be67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:23.822456 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:23.823446 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:23.824333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:23.824333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:23.824333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:23.825147 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 285/540] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:23.828034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a857942-d110-4685-804d-80d658dd93e6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.832338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a857942-d110-4685-804d-80d658dd93e6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.832338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a857942-d110-4685-804d-80d658dd93e6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.832338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 286/541] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:23.835410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a857942-d110-4685-804d-80d658dd93e6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:23.881888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df1ee1-2d2d-4e85-b0cb-c1cc0b10bc44 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:23.908294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:23.923407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c8374-f43f-4429-8203-7023bb6933e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:23.924095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c8374-f43f-4429-8203-7023bb6933e7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.924406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c8374-f43f-4429-8203-7023bb6933e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.924684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c8374-f43f-4429-8203-7023bb6933e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.937855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c8374-f43f-4429-8203-7023bb6933e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:23.939993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c8374-f43f-4429-8203-7023bb6933e7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f48c8374-f43f-4429-8203-7023bb6933e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:23.957490 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:23.958445 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:23.959884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:23.959884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:23.959884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:23.959884 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 256/542] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:23.960845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17fb6b-4e60-47b7-bb1b-940baafa3ecf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:23.960845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17fb6b-4e60-47b7-bb1b-940baafa3ecf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:23.961025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17fb6b-4e60-47b7-bb1b-940baafa3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:23.968405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a857942-d110-4685-804d-80d658dd93e6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:23.978696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0c17fb6b-4e60-47b7-bb1b-940baafa3ecf None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:23.979855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c17fb6b-4e60-47b7-bb1b-940baafa3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:24.005897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9ed50-2f62-41f2-9c4a-3e9de31a1a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.006810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9ed50-2f62-41f2-9c4a-3e9de31a1a39 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.007124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9ed50-2f62-41f2-9c4a-3e9de31a1a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.007391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9ed50-2f62-41f2-9c4a-3e9de31a1a39 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.008020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9ed50-2f62-41f2-9c4a-3e9de31a1a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:24.008327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9ed50-2f62-41f2-9c4a-3e9de31a1a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.009696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9ed50-2f62-41f2-9c4a-3e9de31a1a39 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.022353 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 287/543] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:24.022735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700376e3-8ae8-40a2-869a-d483023f6053 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.023009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700376e3-8ae8-40a2-869a-d483023f6053 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.023556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700376e3-8ae8-40a2-869a-d483023f6053 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.024597 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 288/544] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:24.027893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700376e3-8ae8-40a2-869a-d483023f6053 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.048212 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 257/545] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:24.049068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17fb6b-4e60-47b7-bb1b-940baafa3ecf None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.056451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:24.130010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700376e3-8ae8-40a2-869a-d483023f6053 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.160108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17fb6b-4e60-47b7-bb1b-940baafa3ecf None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.165848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.166452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.166734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.167004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.167476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:24.168040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:24.168542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.170162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.186606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 289/546] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:24.186606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.200161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbd6f93-c7e5-4087-8da1-e38a57fe31c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.200798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6f93-c7e5-4087-8da1-e38a57fe31c9 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.201075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6f93-c7e5-4087-8da1-e38a57fe31c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.201586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbd6f93-c7e5-4087-8da1-e38a57fe31c9 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.201982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd6f93-c7e5-4087-8da1-e38a57fe31c9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.203334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbd6f93-c7e5-4087-8da1-e38a57fe31c9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.220088 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 258/547] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:24 2022] POST /identity/v3/services => generated 251 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:24.225951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a221c5-5e37-46b6-a589-574a6b3349c2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.226585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a221c5-5e37-46b6-a589-574a6b3349c2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.227086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a221c5-5e37-46b6-a589-574a6b3349c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.255010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-67a221c5-5e37-46b6-a589-574a6b3349c2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:24.257652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-67a221c5-5e37-46b6-a589-574a6b3349c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:24.289742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:24.304490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408e5493-acb5-43e5-877c-40ff0969b3fb None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.351262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.352405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.352888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.353303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.356594 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 259/548] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 00:43:24.357452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a221c5-5e37-46b6-a589-574a6b3349c2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.371059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.373086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:24.387668 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:24.388695 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:24.389607 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:24.391634 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 290/549] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:24.392737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.476015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:24.484194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a221c5-5e37-46b6-a589-574a6b3349c2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.524907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608880d2-6068-4552-8639-82d317513489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.525543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608880d2-6068-4552-8639-82d317513489 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.525818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608880d2-6068-4552-8639-82d317513489 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.526088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608880d2-6068-4552-8639-82d317513489 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.531762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a5fff8-3504-4025-a24d-ec4430f480f1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.534374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608880d2-6068-4552-8639-82d317513489 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.535799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608880d2-6068-4552-8639-82d317513489 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-608880d2-6068-4552-8639-82d317513489 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:24.549884 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:24.550837 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:24.551845 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:24.556229 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 260/550] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:24.557184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608880d2-6068-4552-8639-82d317513489 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.578015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.579267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.579748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.580195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.594631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:24.603015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:24.603969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:24.604823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:24.607547 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 291/551] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:24.608813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.613804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:24.672195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608880d2-6068-4552-8639-82d317513489 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.707765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.708570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.708960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.709168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.709862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:24.710164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.711643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.727113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb550f-c914-4eac-a9d6-c4d5c7644dc7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.730983 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 261/552] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:24.731804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.751685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:24.764602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.765213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.765494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.765765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.766498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:24.766801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.768234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.786849 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 292/553] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:24.787717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.855942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10384ccb-9529-441d-94e5-065d06b350b3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.856417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:24.894037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.894919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.895200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.895472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.896128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:24.896243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789364a-3c0a-42ee-bd82-0e0b933d3170 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:24.896462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.897854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.910669 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 262/554] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:24.910669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:24.935870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:24.936505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:24.936789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:24.937063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:24.937714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:24.939075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:24.956449 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 293/555] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:24 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:24.957302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.008270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.037128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f47847-f00d-40d3-8244-a04d0a84de8c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.090830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.091506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.091781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.092051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.104047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.104504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f608180-e168-4af7-9aab-8dcd2dad67b5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.105566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:25.121984 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:25.122992 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:25.123866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:25.123866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:25.123866 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:25.127993 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 263/556] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:25.128980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.134737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.160392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.161029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.161302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.161882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.172488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.173995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:25.191168 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:25.192431 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:25.193318 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:25.193318 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:25.193318 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:25.193318 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 294/557] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:25.193318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.266953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef91419-4b97-42f7-a644-0b45373d6a0b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.297261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.301429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7683b4c-14cf-4863-8411-2b7604900d43 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.311538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.312611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.312897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.313167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.327657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.329583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.337673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.338358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.338698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.339017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.339652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:25.339996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.341447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:25.347041 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:25.349069 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:25.350352 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:25.350352 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:25.350352 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:25.350352 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 264/558] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:25.353054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.358411 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 295/559] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:25.359371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.404170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.476208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf26c95-0f0d-4692-a85e-48a42224a65c None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.502739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86180d12-5cd3-4dfd-8b25-a4135f21544e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.515674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.516287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.516605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.516875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.517577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:25.517847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.519414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.536903 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 296/560] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:25.538612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.541514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.542367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.543546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.543825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.544503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:25.544806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.546164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.560881 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 265/561] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:25.561726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.611011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.655362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.664810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42e34b-2f57-442b-853c-7a4841054b51 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.691363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91b2627-d309-4ac0-9aaa-377c3b8e4003 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.701550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.702471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.702959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.703421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.711111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.712788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:25.724877 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:25.725776 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:25.726665 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:25.729141 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 297/562] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:25.730553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.741079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.741739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.742537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.742824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.753536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.754981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:25.773031 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:25.774003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:25.774999 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:25.778699 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 266/563] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:25.780106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.844681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438be9bd-12e0-474c-a5a1-69f3b4268494 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.880370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.882198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.882719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.883205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.890908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.892738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.902244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:25.906875 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:25.907868 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:25.908872 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:25.909311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d205b-51db-4d96-bef0-706cda96c6a5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:25.911367 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 298/564] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:25.912592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:25.932254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:25.962350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:25.963393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:25.963910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:25.964425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:25.965311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:25.965809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:25.967461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:25.980660 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 267/565] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:25.981733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:26.042349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e698e-3390-44e2-a104-c76a25013a73 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:26.077788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:26.078383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.078664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.078931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.079656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:26.079955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:26.081429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:26.090377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac7966c-7e3c-4973-8e4c-5e980a224f0f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:26.103400 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 299/566] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:26.104266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:26.127210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:26.128183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.128725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.129213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.130145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:26.130657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:26.133221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:26.145531 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 268/567] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:26.146705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:26.166046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:26.179067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:26.211023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f253f2f-456a-4e4e-9029-090bc0eb6c9a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:26.246651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:26.247618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.248098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.248617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/232811b7770447b6a12f4a806a7b0119/roles/d36d481771e04201bdf12458d1795e34` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.254444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663908b0-8ae0-4563-b145-c67415efbaae None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:26.264903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=232811b7770447b6a12f4a806a7b0119, role_id=d36d481771e04201bdf12458d1795e34)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:26.266644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:26.283904 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 300/568] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:26 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/232811b7770447b6a12f4a806a7b0119/roles/d36d481771e04201bdf12458d1795e34 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:26.285106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:26.301416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:26.302127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.302405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.302684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.387666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:26.389119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:26.398016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd7d82-b66b-4ad4-a198-fb2bcd6cd98a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:26.408677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:26.450656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:26.472845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda8313-c11f-4fa4-be3c-536818467a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:26.473588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8313-c11f-4fa4-be3c-536818467a4a None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.473874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8313-c11f-4fa4-be3c-536818467a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.474147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda8313-c11f-4fa4-be3c-536818467a4a None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/0800a1f4140a4d098eadb94eeba922be/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.485588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda8313-c11f-4fa4-be3c-536818467a4a None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=0800a1f4140a4d098eadb94eeba922be, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:26.487071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda8313-c11f-4fa4-be3c-536818467a4a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:26.510043 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 269/569] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:43:26.517339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:26.535830 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 301/570] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:26 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/0800a1f4140a4d098eadb94eeba922be/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:26.537002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198372-1f31-4171-be50-783abea5161c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.537935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198372-1f31-4171-be50-783abea5161c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.538220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8198372-1f31-4171-be50-783abea5161c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.544015 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 302/571] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:26.545788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2e87a-eac5-4274-a19a-85b14bab2c7e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.545997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2e87a-eac5-4274-a19a-85b14bab2c7e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.546341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2e87a-eac5-4274-a19a-85b14bab2c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.566739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-49f2e87a-eac5-4274-a19a-85b14bab2c7e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:26.567829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-49f2e87a-eac5-4274-a19a-85b14bab2c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:26.641883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea86e7-3e28-4452-8c8d-a32157bfb144 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:26.659465 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 303/572] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:26.668404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:26.696109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:26.696812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.697065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.697305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.711910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:26.713377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:26.742142 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:26.743238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:26.744161 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:26.744161 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:26.744161 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:26.744161 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 270/573] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:26.749378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2e87a-eac5-4274-a19a-85b14bab2c7e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:26.883814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2e87a-eac5-4274-a19a-85b14bab2c7e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:26.898777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:26.923616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:26.924202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:26.924509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:26.924780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:26.925421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:26.925719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:26.927070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:26.940985 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 304/574] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:26.946092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:26.984457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:27.070662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70fe16c-e68a-45a0-834d-9af984cfda8b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:27.106396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:27.106971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.107253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.107472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.114681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:27.116258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:27.127785 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:27.128738 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:27.129622 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 271/575] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:27.135372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:27.147998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7f869-e55b-4144-9cab-112e962a6fd1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:27.176745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:27.207612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:27.208099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.208403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.208680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.209840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:27.211218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:27.225419 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:27.226515 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:27.227391 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:27.232941 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 305/576] 10.222.0.39 () {56 vars in 1129 bytes} [Sun Oct 23 00:43:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:27.238454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:27.260419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bd927-da79-48dd-83e3-4c70d016019d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:27.297519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:27.298497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.298968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.299441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.300350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:27.300920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:27.302769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:27.320223 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 272/577] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:27.321287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:27.336937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:27.440372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e77f8ab-8565-4642-8505-9a6c8f267747 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:27.492793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:27.495214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190a3f86-2b38-487d-acdb-5ee6af39be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:27.495847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a3f86-2b38-487d-acdb-5ee6af39be56 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.496122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a3f86-2b38-487d-acdb-5ee6af39be56 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.496415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190a3f86-2b38-487d-acdb-5ee6af39be56 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.497126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a3f86-2b38-487d-acdb-5ee6af39be56 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:27.497429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a3f86-2b38-487d-acdb-5ee6af39be56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:27.498865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190a3f86-2b38-487d-acdb-5ee6af39be56 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:27.509774 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 306/578] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Oct 23 00:43:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:27.510503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dce154-b18b-408c-8385-c8cb00e3441a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:27.511272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f07f9f-fcec-44db-bcd6-bfe448e0b30c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.511573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f07f9f-fcec-44db-bcd6-bfe448e0b30c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.511813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f07f9f-fcec-44db-bcd6-bfe448e0b30c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.517924 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 307/579] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:27.517924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f07f9f-fcec-44db-bcd6-bfe448e0b30c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:27.564173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df6fc8-0106-4bf2-8bfd-d8619bafb784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:27.565380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df6fc8-0106-4bf2-8bfd-d8619bafb784 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.565814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df6fc8-0106-4bf2-8bfd-d8619bafb784 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.566237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df6fc8-0106-4bf2-8bfd-d8619bafb784 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.567377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df6fc8-0106-4bf2-8bfd-d8619bafb784 None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=f04fb9e1d1bd42ac815078b61f6a17f8) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:27.568330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df6fc8-0106-4bf2-8bfd-d8619bafb784 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:27.570764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df6fc8-0106-4bf2-8bfd-d8619bafb784 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:27.579403 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 273/580] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:27 2022] GET /identity/v3/role_assignments?user.id=f04fb9e1d1bd42ac815078b61f6a17f8&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:27.580767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709be95-889c-4df0-b70d-8d97283b9b41 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.581187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709be95-889c-4df0-b70d-8d97283b9b41 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.581597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709be95-889c-4df0-b70d-8d97283b9b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.603651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5709be95-889c-4df0-b70d-8d97283b9b41 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:27.606063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5709be95-889c-4df0-b70d-8d97283b9b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:27.676856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:27.682029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f07f9f-fcec-44db-bcd6-bfe448e0b30c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:27.696429 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 274/581] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:27.744654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:27.745282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.745687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.745968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.746655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:27.747032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:27.748504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:27.758934 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 308/582] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Oct 23 00:43:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:27.764863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709be95-889c-4df0-b70d-8d97283b9b41 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:27.875466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:27.896366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709be95-889c-4df0-b70d-8d97283b9b41 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:27.939812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:27.940459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:27.940739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:27.941009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:27.941720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] RBAC: Adding query filter params (interface=public, service_id=f3fb98225aa64eccaa461e6e92441647, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:27.942028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:27.943934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:27.952227 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 275/583] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:43:27 2022] GET /identity/v3/endpoints?service_id=f3fb98225aa64eccaa461e6e92441647&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:27.957435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:28.060620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:28.066591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b62d6c1-a477-449a-b166-c98d2340f402 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:28.106854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:28.107473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:28.107752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:28.108025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:28.108727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:28.110026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:28.120598 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 309/584] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:27 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:28.177667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca641-a820-4500-9886-2cb0361a1445 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:28.215905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:28.216913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:28.217396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:28.217920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] PATH_INFO: `/v3/services/network` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:28.218781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:28.220437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:28.230864 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:28.232003 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:28.232939 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:28.239576 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 276/585] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:28 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:28.243054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:28.280233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:28.368278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55faf159-d95d-4246-9c55-20014436a29a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:28.419760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:28.420571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:28.420956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:28.421329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:28.422139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] RBAC: Adding query filter params (name=network) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:28.422559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:28.424047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:28.432572 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 310/586] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:28.438308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:28.452801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:28.543859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2106c37-47ba-44b3-888d-af5487b682fa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:28.581763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:28.582528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:28.582901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:28.583290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:28.584046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:28.584515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:28.585971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:28.591410 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 277/587] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:28.596772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:28.711839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:28.721404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934a0ce-e941-4278-ab3e-e253e40ed51f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:28.759873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510f5f-bbca-4689-bf83-3551ebcb8dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:28.760699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510f5f-bbca-4689-bf83-3551ebcb8dbe None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:28.761072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510f5f-bbca-4689-bf83-3551ebcb8dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:28.761441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510f5f-bbca-4689-bf83-3551ebcb8dbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:28.762244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510f5f-bbca-4689-bf83-3551ebcb8dbe None admin] RBAC: Adding query filter params (interface=public, service_id=66bebc202b254153aa66ac4273474171, region_id=RegionOne) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:28.762649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510f5f-bbca-4689-bf83-3551ebcb8dbe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:28.764122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510f5f-bbca-4689-bf83-3551ebcb8dbe None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:28.772382 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 311/588] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:43:28 2022] GET /identity/v3/endpoints?service_id=66bebc202b254153aa66ac4273474171&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:28.781543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:28.880675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:28.905152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a29d0-85eb-4584-91bc-a0821c0649a7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:28.942362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07996ddc-2f4d-47b6-969e-ee4e96ef1c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:28.943008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07996ddc-2f4d-47b6-969e-ee4e96ef1c23 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:28.943294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07996ddc-2f4d-47b6-969e-ee4e96ef1c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:28.943595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07996ddc-2f4d-47b6-969e-ee4e96ef1c23 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:28.944202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07996ddc-2f4d-47b6-969e-ee4e96ef1c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:28.945595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07996ddc-2f4d-47b6-969e-ee4e96ef1c23 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:28.951179 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 278/589] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:28 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:29.031584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a592508-9458-4b44-958a-e8317b9804a8 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:29.031968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a592508-9458-4b44-958a-e8317b9804a8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:29.032153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a592508-9458-4b44-958a-e8317b9804a8 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:29.033872 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 312/590] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:29.040855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef96005-9133-4edc-adcb-b6c9ea3d63fb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:29.041198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef96005-9133-4edc-adcb-b6c9ea3d63fb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:29.041407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef96005-9133-4edc-adcb-b6c9ea3d63fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:29.061657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1ef96005-9133-4edc-adcb-b6c9ea3d63fb None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:29.062938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1ef96005-9133-4edc-adcb-b6c9ea3d63fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:29.143276 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 279/591] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:29.180834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821981-25ff-45d3-bd66-9f5a91d9b734 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:29.181200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821981-25ff-45d3-bd66-9f5a91d9b734 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:29.181447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9821981-25ff-45d3-bd66-9f5a91d9b734 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:29.182978 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 313/592] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:29.188703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6ba1a-fe56-4ae4-8f06-39c3c4bad3ec None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:29.188978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6ba1a-fe56-4ae4-8f06-39c3c4bad3ec None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:29.189252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f6ba1a-fe56-4ae4-8f06-39c3c4bad3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:29.213882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-43f6ba1a-fe56-4ae4-8f06-39c3c4bad3ec None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:29.215144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-43f6ba1a-fe56-4ae4-8f06-39c3c4bad3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:29.282294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:29.288914 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 280/593] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:29.485594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821981-25ff-45d3-bd66-9f5a91d9b734 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:29.601205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9821981-25ff-45d3-bd66-9f5a91d9b734 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:29.601591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6ba1a-fe56-4ae4-8f06-39c3c4bad3ec None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:29.645319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:29.646503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:29.647494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:29.647958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:29.658525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:29.663898 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:29.664935 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:43:29.665843 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:29.668424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 314/594] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:43:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:29.674838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:29.724743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:29.787270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6433578-25e2-4202-bfbd-342376dd9bd8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:29.825468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:29.826800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:29.827284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:29.827767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:29.828675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:29.829185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:29.830798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:29.840896 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 315/595] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:43:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:29.848693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:29.866571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f6ba1a-fe56-4ae4-8f06-39c3c4bad3ec None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:29.933003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:29.933637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:29.933918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:29.934188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:29.954690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:29.959483 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:29.960546 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:43:29.961389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:29.969125 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 281/596] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:29.974244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.008683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:30.042590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd4a2a-e587-4bb9-af31-28e2de852dff None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:30.087166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:30.119227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:30.119872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.120149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.120445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.142201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:30.143648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:30.159757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09dc1-320b-427f-82bc-f9a8f85afc2e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:30.169438 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:30.170430 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:30.171331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:30.171331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:30.171331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:30.171331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 316/597] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:30.187787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.210744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:30.211366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.211642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.211913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.212617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:30.212920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:30.214286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:30.231930 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 282/598] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:30.237740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.336239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f9d0d-dab8-42c6-b534-447274f5f82e None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:30.373265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:30.374430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.375225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.375691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.376691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:30.377280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:30.378915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:30.395753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:30.396369 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 317/599] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:30.398740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.477188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaabd5-7b64-4f02-87a2-95be0c56f11d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:30.520743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:30.550533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39919262-a0f7-47a4-8746-8c350edb9f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:30.551532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39919262-a0f7-47a4-8746-8c350edb9f6a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.552023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39919262-a0f7-47a4-8746-8c350edb9f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.553699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39919262-a0f7-47a4-8746-8c350edb9f6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.574012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39919262-a0f7-47a4-8746-8c350edb9f6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:30.576737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39919262-a0f7-47a4-8746-8c350edb9f6a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:30.594435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce1b2a-42ea-436a-84a2-ceccc1fdf5b6 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-39919262-a0f7-47a4-8746-8c350edb9f6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:30.607325 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:30.608324 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:30.609249 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:30.609249 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:30.609249 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:30.616342 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 283/600] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:30.617356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259035-46b6-49e4-8561-0ac682669e15 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.617633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259035-46b6-49e4-8561-0ac682669e15 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.617902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8259035-46b6-49e4-8561-0ac682669e15 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.619297 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 284/601] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:30.619968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259035-46b6-49e4-8561-0ac682669e15 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.643517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df092380-e428-4e65-a08a-eb6e2aad8130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:30.644625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df092380-e428-4e65-a08a-eb6e2aad8130 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.645140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df092380-e428-4e65-a08a-eb6e2aad8130 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.645612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df092380-e428-4e65-a08a-eb6e2aad8130 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.654329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df092380-e428-4e65-a08a-eb6e2aad8130 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:30.656169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df092380-e428-4e65-a08a-eb6e2aad8130 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-df092380-e428-4e65-a08a-eb6e2aad8130 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:30.670051 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:30.671088 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:30.672034 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:30.674815 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 318/602] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:43:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:30.680645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a8ef-ed93-400f-84c8-7915791781d4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.681324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a8ef-ed93-400f-84c8-7915791781d4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.682075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7a8ef-ed93-400f-84c8-7915791781d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.703340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-01b7a8ef-ed93-400f-84c8-7915791781d4 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:30.705201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-01b7a8ef-ed93-400f-84c8-7915791781d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:30.731538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8259035-46b6-49e4-8561-0ac682669e15 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:30.760041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:30.773001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:30.773873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.774308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.774754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.775629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:30.776116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:30.777655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:30.784355 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 319/603] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:30.784889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7a8ef-ed93-400f-84c8-7915791781d4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.799535 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 285/604] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:30.804952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.884203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:30.901115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7a8ef-ed93-400f-84c8-7915791781d4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:30.936878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf92e1-cd2d-44e3-a04a-19216dfd69ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:30.937869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf92e1-cd2d-44e3-a04a-19216dfd69ea None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.938153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf92e1-cd2d-44e3-a04a-19216dfd69ea None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.938483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf92e1-cd2d-44e3-a04a-19216dfd69ea None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.939213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf92e1-cd2d-44e3-a04a-19216dfd69ea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:30.939514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf92e1-cd2d-44e3-a04a-19216dfd69ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:30.940987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf92e1-cd2d-44e3-a04a-19216dfd69ea None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:30.965177 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 320/605] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Oct 23 00:43:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:30.965177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6b110-f67a-42bc-949f-174527893c50 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:30.965177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6b110-f67a-42bc-949f-174527893c50 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:30.965177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6b110-f67a-42bc-949f-174527893c50 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:30.965177 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 321/606] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:30.965177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6b110-f67a-42bc-949f-174527893c50 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:30.977082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10233e5-2e21-4d13-acab-522f33727bd8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.018573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.022298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438d78d-59b8-4f42-895a-972ccf9534a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.023064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d78d-59b8-4f42-895a-972ccf9534a1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.023349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d78d-59b8-4f42-895a-972ccf9534a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.023620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438d78d-59b8-4f42-895a-972ccf9534a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.033189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d78d-59b8-4f42-895a-972ccf9534a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.034628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438d78d-59b8-4f42-895a-972ccf9534a1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2438d78d-59b8-4f42-895a-972ccf9534a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:31.048513 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:31.049503 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.050432 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 286/607] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:43:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:31.051036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb075d34-6739-4019-a119-7a12bc67ea6a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.051640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb075d34-6739-4019-a119-7a12bc67ea6a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.051640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb075d34-6739-4019-a119-7a12bc67ea6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.072731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cb075d34-6739-4019-a119-7a12bc67ea6a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:31.074052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cb075d34-6739-4019-a119-7a12bc67ea6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:31.094027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6b110-f67a-42bc-949f-174527893c50 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.131942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.132619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.132905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.133177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.152430 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 287/608] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:31.152430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb075d34-6739-4019-a119-7a12bc67ea6a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.176974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.230465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.232369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.273560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb075d34-6739-4019-a119-7a12bc67ea6a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.280849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.315791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.316672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.317098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.317543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.319495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:31.319957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.322042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.343908 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 288/609] 10.222.0.39 () {56 vars in 1135 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:31.344901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.351237 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 322/610] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:43:31.352315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.443870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.466661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2a878d-d1fa-4ac5-b70e-dc5f5485db1e None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.472541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f856515-9e05-4b1a-b5b5-16b3cae77b66 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.505164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.506440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.507022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.507261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.507899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.509350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.510092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.510731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.511007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.511270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:31.519140 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:31.520210 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.521426 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 289/611] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:31.522278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.527383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.589906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.592205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.626809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd158d-9d5b-4bd6-922b-3d543bcab94b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.659143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.659765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.660043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.660332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.671206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.672675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.675305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:31.689089 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:31.690259 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:31.696132 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:31.696132 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:31.696132 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.696132 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 290/612] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:31.696399 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 323/613] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:43:31.696487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.696566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.733008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.789711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970838f6-63e5-4432-91ab-7f9404929759 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.791123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f4844-5a03-425f-a0fe-df8d8f846128 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.823728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10feb130-6847-4df0-b456-1bacea3076ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.824702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feb130-6847-4df0-b456-1bacea3076ae None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.824702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feb130-6847-4df0-b456-1bacea3076ae None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.824888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10feb130-6847-4df0-b456-1bacea3076ae None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.825475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feb130-6847-4df0-b456-1bacea3076ae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:31.825776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feb130-6847-4df0-b456-1bacea3076ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.827113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10feb130-6847-4df0-b456-1bacea3076ae None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.828523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.829156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.829424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.829697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.830279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.831627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.831747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 324/614] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:31.832846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e938b-547c-4133-9628-b907a5760ad3 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.833122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e938b-547c-4133-9628-b907a5760ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.833391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008e938b-547c-4133-9628-b907a5760ad3 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.834846 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 325/615] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:31.835536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e938b-547c-4133-9628-b907a5760ad3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:31.842051 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:31.843238 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:31.844405 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 291/616] 10.222.0.39 () {56 vars in 1129 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:31.845238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.902251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.937751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008e938b-547c-4133-9628-b907a5760ad3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.939225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:31.942407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4fbc8c-4f6e-49d0-b699-5c782cea5db1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:31.972656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.973417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.973731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.973995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.975229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:31.975521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.976562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2c501-6469-4e2f-baa5-319c94fac23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:31.976899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:31.977192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2c501-6469-4e2f-baa5-319c94fac23e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:31.977468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2c501-6469-4e2f-baa5-319c94fac23e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:31.977739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2c501-6469-4e2f-baa5-319c94fac23e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:31.987728 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 326/617] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:31.988523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:31.993295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2c501-6469-4e2f-baa5-319c94fac23e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:31.994725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2c501-6469-4e2f-baa5-319c94fac23e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-4bc2c501-6469-4e2f-baa5-319c94fac23e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:32.009825 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:32.011061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:32.011953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:32.011953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:32.011953 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:32.011953 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 292/618] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:32.012391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8fd9-9cef-4ba5-8c7d-fb353101a925 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.012668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8fd9-9cef-4ba5-8c7d-fb353101a925 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.012939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0f8fd9-9cef-4ba5-8c7d-fb353101a925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.031449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5f0f8fd9-9cef-4ba5-8c7d-fb353101a925 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:32.032667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f0f8fd9-9cef-4ba5-8c7d-fb353101a925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:32.087758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca78f745-df1d-498e-a532-6f11483c10a4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.097448 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 293/619] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:32.098258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f8fd9-9cef-4ba5-8c7d-fb353101a925 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.123208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.124010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.124508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.124944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.125568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:32.125866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.127208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.132032 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 327/620] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:32.132720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.144584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.170809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.231269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0f8fd9-9cef-4ba5-8c7d-fb353101a925 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.263943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443b1cc-247e-4aa4-b3af-690b02d938a3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.282791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.283567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.283850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.284125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.284806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:32.285110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.287878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.294611 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 294/621] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:32.295994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.308478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.309059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.309343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.309619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.320286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.322171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:32.335323 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:32.336280 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:32.337439 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:32.339191 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 328/622] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:32.340043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.410651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1b3fd8-a651-48c9-b103-b2224249ddd4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.418453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.436275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.448224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.448862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.449141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.449519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.450046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:32.450403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.451685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.452579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343026-ad6b-402c-8f95-ef3b93a2c358 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.468880 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 295/623] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:32.468880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.491118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.491736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.492074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.492439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.493117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:32.493480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.494920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.500261 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 329/624] 10.222.0.39 () {56 vars in 1124 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:32.501250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.580000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720c23c-4402-468f-9408-056fec4c2dfd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.605019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8fedcc-80d6-419e-84b0-24c079e75778 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.616885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.617738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.618218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.618767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.619462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:32.619772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.621344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.636364 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 296/625] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:32.636916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.646304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.646969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.647247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.647515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] PATH_INFO: `/v3/services/network` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.648092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.649489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:32.658108 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:32.659094 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:32.660791 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 330/626] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:32.661569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.667134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.682818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.750027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2d1b6-d3c8-469e-8699-fd6e18b3b921 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.772945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74418504-ca9e-4039-b153-6b2e48717eff None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.789546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.790182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.790461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.790731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.797950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.799712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.807905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:32.808542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:32.808825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:32.809150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:32.811805 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:32.812886 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:32.814094 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:32.815048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 297/627] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:32.815550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:32.815889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.817408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:32.845794 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 331/628] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:32.846595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:32.930303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.950115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:32.951806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4609cad-b8f8-4dc6-ab8e-a933bc551711 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:32.997806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad662fca-a918-44a4-b147-c8705227ccd7 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:33.007204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359830b3-1928-42dc-878b-d02496787fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:33.008148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359830b3-1928-42dc-878b-d02496787fea None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:33.008607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359830b3-1928-42dc-878b-d02496787fea None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:33.009025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359830b3-1928-42dc-878b-d02496787fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:33.010415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359830b3-1928-42dc-878b-d02496787fea None admin] RBAC: Adding query filter params (role.id=d36d481771e04201bdf12458d1795e34, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=232811b7770447b6a12f4a806a7b0119) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:33.010909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359830b3-1928-42dc-878b-d02496787fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:33.013238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359830b3-1928-42dc-878b-d02496787fea None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:33.029144 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 298/629] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/role_assignments?user.id=232811b7770447b6a12f4a806a7b0119&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=d36d481771e04201bdf12458d1795e34 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:33.029144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359830b3-1928-42dc-878b-d02496787fea None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:33.047250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:33.047962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:33.048268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:33.049815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:33.050537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] RBAC: Adding query filter params (name=network) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:33.050846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:33.052396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:33.061584 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 332/630] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:33.067937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:33.165601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359830b3-1928-42dc-878b-d02496787fea None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:33.217877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc3cf86-a96f-422c-a97d-28c4bac6dc62 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:33.223771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:33.224759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:33.225207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:33.225623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:33.226579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:33.227027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:33.228757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:33.254350 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 299/631] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:33.261996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:33.270637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:33.278203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:33.279379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:33.280329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:33.280819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:33.281240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:33.282023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] RBAC: Adding query filter params (type=network) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:33.282469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:33.284907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:33.290906 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 333/632] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:33.296264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:33.399391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c48f3d-f3de-4cc6-9418-83da3148e7a3 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:33.413657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f067e6-eb67-4ff5-a35d-0647e83dce73 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:33.445780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e77b79-72ad-433e-8b73-91c079d93807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:33.446714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e77b79-72ad-433e-8b73-91c079d93807 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:33.447137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e77b79-72ad-433e-8b73-91c079d93807 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:33.447550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e77b79-72ad-433e-8b73-91c079d93807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:33.448445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e77b79-72ad-433e-8b73-91c079d93807 None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=0800a1f4140a4d098eadb94eeba922be) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:33.448914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e77b79-72ad-433e-8b73-91c079d93807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:33.450486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e77b79-72ad-433e-8b73-91c079d93807 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:33.457540 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 300/633] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:33 2022] GET /identity/v3/role_assignments?user.id=0800a1f4140a4d098eadb94eeba922be&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:33.463941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624bcbb-6c3c-43bc-960e-535dfdaa44ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:33.464922 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624bcbb-6c3c-43bc-960e-535dfdaa44ab None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:33.465367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624bcbb-6c3c-43bc-960e-535dfdaa44ab None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:33.465784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624bcbb-6c3c-43bc-960e-535dfdaa44ab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:33.466775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624bcbb-6c3c-43bc-960e-535dfdaa44ab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:33.468434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624bcbb-6c3c-43bc-960e-535dfdaa44ab None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:33.510072 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 334/634] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:34.454780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0f31-b86a-4d38-ac04-697303c2b9db None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:34.455851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0f31-b86a-4d38-ac04-697303c2b9db None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:34.456369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c0f31-b86a-4d38-ac04-697303c2b9db None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:34.458221 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 301/635] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:34.463862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16870e64-585e-4089-b0dc-afd299c20038 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:34.464759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16870e64-585e-4089-b0dc-afd299c20038 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:34.465232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16870e64-585e-4089-b0dc-afd299c20038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:34.484873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-16870e64-585e-4089-b0dc-afd299c20038 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:34.486438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-16870e64-585e-4089-b0dc-afd299c20038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:34.538454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:34.566636 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 335/636] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:34.835976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0f31-b86a-4d38-ac04-697303c2b9db None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:34.917857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:34.945471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c0f31-b86a-4d38-ac04-697303c2b9db None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:34.980751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460b822-2a34-42ba-994d-04200f360ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:34.981365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b822-2a34-42ba-994d-04200f360ea2 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:34.981643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b822-2a34-42ba-994d-04200f360ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:34.981911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460b822-2a34-42ba-994d-04200f360ea2 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:34.982559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460b822-2a34-42ba-994d-04200f360ea2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:34.984025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460b822-2a34-42ba-994d-04200f360ea2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:35.004473 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 302/637] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:34 2022] POST /identity/v3/services => generated 269 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:35.346316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e51c4f-8fe1-47f6-8771-428cc04bec0c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:35.346728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e51c4f-8fe1-47f6-8771-428cc04bec0c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:35.346905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e51c4f-8fe1-47f6-8771-428cc04bec0c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:35.348607 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 336/638] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:35.354331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed221c-ea46-4323-a1ee-79e0e14919d8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:35.354637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed221c-ea46-4323-a1ee-79e0e14919d8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:35.354891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed221c-ea46-4323-a1ee-79e0e14919d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:35.373003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-53ed221c-ea46-4323-a1ee-79e0e14919d8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:35.374181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-53ed221c-ea46-4323-a1ee-79e0e14919d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:35.448544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf34c9c-e2b7-4a0a-bb70-f10849c3f715 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:35.448866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf34c9c-e2b7-4a0a-bb70-f10849c3f715 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:35.449090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf34c9c-e2b7-4a0a-bb70-f10849c3f715 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:35.450656 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 337/639] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:35.451413 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 303/640] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:35.455575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33981e-6f63-430c-a966-17b5c85ab1cd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:35.455854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33981e-6f63-430c-a966-17b5c85ab1cd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:35.456122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab33981e-6f63-430c-a966-17b5c85ab1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:35.474273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ab33981e-6f63-430c-a966-17b5c85ab1cd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:35.476085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ab33981e-6f63-430c-a966-17b5c85ab1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:35.547781 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 338/641] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:35.730453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed221c-ea46-4323-a1ee-79e0e14919d8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:35.794963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:35.821371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77213b3-6606-4b5c-b1d9-866017044fb5 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:35.821972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77213b3-6606-4b5c-b1d9-866017044fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:35.822437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77213b3-6606-4b5c-b1d9-866017044fb5 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:35.824213 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 339/642] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:35.830171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129d0ec-ac40-402b-a0a2-0b13118b05c0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:35.830741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129d0ec-ac40-402b-a0a2-0b13118b05c0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:35.831187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129d0ec-ac40-402b-a0a2-0b13118b05c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:35.837238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed221c-ea46-4323-a1ee-79e0e14919d8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:35.850597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f129d0ec-ac40-402b-a0a2-0b13118b05c0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:35.852055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f129d0ec-ac40-402b-a0a2-0b13118b05c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:35.875019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:35.875647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:35.875932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:35.876201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:35.876806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:35.878174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:35.887114 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:35.888136 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:35.889032 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 304/643] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:35.889830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:35.929246 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 340/644] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:35.930528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129d0ec-ac40-402b-a0a2-0b13118b05c0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:35.977360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.010880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655d3d25-ae88-497a-b2a8-21d62a42b895 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.038076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.053008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.053992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.054555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.055054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.063349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129d0ec-ac40-402b-a0a2-0b13118b05c0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.066501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:36.072533 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:36.073446 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:43:36.074270 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:36.076620 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 305/645] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Oct 23 00:43:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:36.082338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.103328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.103898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.104178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.104541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.105165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:36.105467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.106887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.111599 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 341/646] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:36.117004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.202855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6545b3-6b5b-447b-a141-b8ea5cbdd1a0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.231952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.235297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dee3ea-4a3a-421b-aa6a-c6068e714f0d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.243890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.244523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.244811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.245074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.245726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:36.246025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.247380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.258436 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 306/647] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:36.259233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.275696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.276667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.277145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.277611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.278425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:36.278932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.280377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.287179 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 342/648] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:36.287791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.321250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.391687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c11dbd-95d2-4018-a617-43e110ba711f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.404791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f9067a-b8a0-4f6a-9747-f7d45bce3029 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.436804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.437482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.437759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.438027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.438611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.440013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.448964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.450064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.450540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:36.450791 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:36.451723 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:36.452603 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 307/649] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:36.453395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.457914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.470408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.471913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:36.490823 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:36.492220 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:36.493141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:36.493141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:36.493141 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:36.493141 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 343/650] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:36.497381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.516741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.589661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4247eca-8107-49e1-b9cb-0b8100b9aa40 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.597655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7221e5-fa52-45d3-b7c7-a1677a9da472 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.603110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.635259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.636214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.636706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.637171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.638156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:36.638629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.639235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.639908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.640250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.640686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.640948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.641446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:36.641810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.643404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.646713 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 308/651] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:36.655768 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 344/652] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:36.656930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.661508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.753711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.778402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723e07a-4096-40f4-9998-86f7210fa1b2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.783923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178725d-c3dd-4d18-aa8c-51562145b6c4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:36.822180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.823067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.823350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.823626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.824259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:36.824588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.827806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.832654 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 309/653] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:36.836903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:36.837880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:36.838402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:36.838885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:36.847426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:36.849992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:36.863536 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:36.864582 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:36.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:36.868758 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 345/654] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:36.874232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:36.919460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:36.989694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b7844-f32c-4c67-82c5-31aed106059d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:37.029196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:37.029978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.030260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.030528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.032392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:37.032392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:37.033009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:37.054079 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 310/655] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Oct 23 00:43:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:37.059820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:37.170272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108470e-f3bf-4549-99f7-68c23663323f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:37.209457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3458a3f0-992e-47f1-8142-005b29f01483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:37.210362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458a3f0-992e-47f1-8142-005b29f01483 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.210808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458a3f0-992e-47f1-8142-005b29f01483 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.211222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458a3f0-992e-47f1-8142-005b29f01483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.214683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:37.223661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458a3f0-992e-47f1-8142-005b29f01483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:37.225834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3458a3f0-992e-47f1-8142-005b29f01483 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-3458a3f0-992e-47f1-8142-005b29f01483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:37.241082 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:37.242079 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:37.243118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:37.243118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:37.243118 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:37.243118 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 346/656] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:37.249563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:37.317035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c95d-5959-4346-9007-ca62986b1990 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.317035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c95d-5959-4346-9007-ca62986b1990 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.317035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503c95d-5959-4346-9007-ca62986b1990 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.317960 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 347/657] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:37.323236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02df036-e6b2-4c09-90d8-2537d043bcab None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.323531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02df036-e6b2-4c09-90d8-2537d043bcab None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.323793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02df036-e6b2-4c09-90d8-2537d043bcab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.345462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b02df036-e6b2-4c09-90d8-2537d043bcab None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:37.346964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b02df036-e6b2-4c09-90d8-2537d043bcab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:37.357945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:37.363712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9618a-ea9d-4716-bd8b-b0fc66e39c46 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:37.402654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:37.403163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.403444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.403719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.404649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:37.404957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:37.407099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:37.421173 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 311/658] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:37.429579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:37.436372 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 348/659] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:37 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:37.533923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf133d7-5ede-4f82-9ac9-56d313385ebd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:37.568961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:37.569949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.570407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.570853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.577940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:37.579676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:37.591869 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:37.592790 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:37.593872 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 312/660] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:37.601593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02df036-e6b2-4c09-90d8-2537d043bcab None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:37.699676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:37.706914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02df036-e6b2-4c09-90d8-2537d043bcab None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:37.720416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:37.726819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:37.746241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:37.746854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.747129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.747398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.748087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:37.748413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:37.750028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:37.767904 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 349/661] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:37.769224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:37.815879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cd75f-4889-4806-9071-d06a30900685 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:37.865298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:37.866428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.867031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.867506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.868353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:37.869955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:37.877500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda6c433-11c8-478b-b756-1c91b25be03a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:37.882286 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:37.883254 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:37.884143 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 313/662] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Oct 23 00:43:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:37.891773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:37.922523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:37.923292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:37.923562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:37.923838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/232811b7770447b6a12f4a806a7b0119/roles/eda5bb304e03428dbc184a9019dc3cda` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:37.926246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=232811b7770447b6a12f4a806a7b0119, role_id=eda5bb304e03428dbc184a9019dc3cda)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:37.928520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:37.952020 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 350/663] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:37 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/232811b7770447b6a12f4a806a7b0119/roles/eda5bb304e03428dbc184a9019dc3cda => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:43:37.994222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:38.017911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aabc6cb-d5ac-4026-9144-7531d2153408 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:38.056962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:38.057639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:38.057922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:38.058192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:38.058830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:38.059130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:38.060589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:38.066329 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 314/664] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:38.072421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:38.117085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:38.181343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b081a66-8cc5-4994-8c6e-f6f4428700fc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:38.216761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:38.217293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:38.217570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:38.217840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:38.218467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:38.218768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:38.220121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:38.225543 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 351/665] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:38.231129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:38.337566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e98c-1402-4320-bc92-d459312a060a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:38.373126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bea93-2199-4dd2-a8a7-c8961b6fe2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:38.373744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bea93-2199-4dd2-a8a7-c8961b6fe2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:38.374020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bea93-2199-4dd2-a8a7-c8961b6fe2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:38.374290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bea93-2199-4dd2-a8a7-c8961b6fe2a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:38.374997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bea93-2199-4dd2-a8a7-c8961b6fe2a9 None admin] RBAC: Adding query filter params (interface=public, service_id=a9d5844db3cb4db087591e3ecd50806b, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:38.375303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bea93-2199-4dd2-a8a7-c8961b6fe2a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:38.376740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bea93-2199-4dd2-a8a7-c8961b6fe2a9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:38.382763 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 315/666] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:43:38 2022] GET /identity/v3/endpoints?service_id=a9d5844db3cb4db087591e3ecd50806b&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:38.388287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:38.504290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df63ee-ac1c-4e44-a5aa-9ff71f35abab None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:38.505805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:38.540678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fb0fd-0460-40ee-9b45-92ce4b438de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:38.541300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fb0fd-0460-40ee-9b45-92ce4b438de5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:38.541579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fb0fd-0460-40ee-9b45-92ce4b438de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:38.541854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fb0fd-0460-40ee-9b45-92ce4b438de5 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:38.542459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fb0fd-0460-40ee-9b45-92ce4b438de5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:38.543812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79fb0fd-0460-40ee-9b45-92ce4b438de5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:38.549897 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 352/667] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:38 2022] GET /identity/v3/services => generated 1029 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:43:38.570082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f1494-23bf-45d1-af56-a95b0259372d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:38.570736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f1494-23bf-45d1-af56-a95b0259372d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:38.571223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f1494-23bf-45d1-af56-a95b0259372d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:38.573097 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 316/668] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:38.582213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b38b8d-14e5-4494-a6f1-c76317134cd6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:38.582844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b38b8d-14e5-4494-a6f1-c76317134cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:38.583310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b38b8d-14e5-4494-a6f1-c76317134cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:38.603541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-21b38b8d-14e5-4494-a6f1-c76317134cd6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:38.605621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-21b38b8d-14e5-4494-a6f1-c76317134cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:38.688040 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 353/669] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:38.969529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f1494-23bf-45d1-af56-a95b0259372d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:38.981411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:39.074851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f1494-23bf-45d1-af56-a95b0259372d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:39.109665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800ee187-d618-4420-8ae3-10b6171b4c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:39.110185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ee187-d618-4420-8ae3-10b6171b4c46 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:39.110461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ee187-d618-4420-8ae3-10b6171b4c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:39.110729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800ee187-d618-4420-8ae3-10b6171b4c46 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:39.111381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ee187-d618-4420-8ae3-10b6171b4c46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:39.112757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800ee187-d618-4420-8ae3-10b6171b4c46 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:39.129747 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 317/670] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:38 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:39.271052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9437d4-2c77-48e8-837e-c256c3eb4ea9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:39.271465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9437d4-2c77-48e8-837e-c256c3eb4ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:39.271653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9437d4-2c77-48e8-837e-c256c3eb4ea9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:39.273391 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 354/671] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:39.279373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea7f97-7831-48a3-9044-321c2a04e091 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:39.279674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea7f97-7831-48a3-9044-321c2a04e091 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:39.279928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea7f97-7831-48a3-9044-321c2a04e091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:39.299989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-57ea7f97-7831-48a3-9044-321c2a04e091 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:39.301224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-57ea7f97-7831-48a3-9044-321c2a04e091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:39.369613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:39.388654 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 318/672] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:39 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:39.651867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9437d4-2c77-48e8-837e-c256c3eb4ea9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:39.750195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:39.758021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9437d4-2c77-48e8-837e-c256c3eb4ea9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:39.793183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397a0c8-58ca-4a76-ad65-3ed6064de164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:39.793791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a0c8-58ca-4a76-ad65-3ed6064de164 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:39.794072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a0c8-58ca-4a76-ad65-3ed6064de164 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:39.794388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397a0c8-58ca-4a76-ad65-3ed6064de164 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:39.795057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397a0c8-58ca-4a76-ad65-3ed6064de164 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:39.796446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397a0c8-58ca-4a76-ad65-3ed6064de164 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:39.820480 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 355/673] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:39 2022] POST /identity/v3/services => generated 241 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:40.260488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7deaf-c8f8-4858-ac86-ca619a2981b9 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:40.261017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7deaf-c8f8-4858-ac86-ca619a2981b9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:40.261017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7deaf-c8f8-4858-ac86-ca619a2981b9 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:40.262598 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 319/674] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:40.268004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0140b-1afd-4415-b8a3-ab45b2cf24a0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:40.268315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0140b-1afd-4415-b8a3-ab45b2cf24a0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:40.268602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0140b-1afd-4415-b8a3-ab45b2cf24a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:40.286830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-18e0140b-1afd-4415-b8a3-ab45b2cf24a0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:40.288076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-18e0140b-1afd-4415-b8a3-ab45b2cf24a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:40.368747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 356/675] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:40.639797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7deaf-c8f8-4858-ac86-ca619a2981b9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:40.753603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7deaf-c8f8-4858-ac86-ca619a2981b9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:40.789480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:40.790537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:40.791015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:40.791482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:40.800100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23978947-0ed8-4f50-a203-f9fe39aa66d6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:40.801659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23978947-0ed8-4f50-a203-f9fe39aa66d6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:40.802157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:40.803033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23978947-0ed8-4f50-a203-f9fe39aa66d6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:40.804891 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 357/676] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:40.808332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:40.809278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:40.810163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 320/677] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Oct 23 00:43:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:40.813434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37f6ec-e372-444f-b85c-3118db2b0c78 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:40.813776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37f6ec-e372-444f-b85c-3118db2b0c78 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:40.814113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37f6ec-e372-444f-b85c-3118db2b0c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:40.815100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:40.835079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1b37f6ec-e372-444f-b85c-3118db2b0c78 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:40.836618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1b37f6ec-e372-444f-b85c-3118db2b0c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:40.882997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:40.919008 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 358/678] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:40.942059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a012b46-9735-47f5-8d24-ffc64f1bda8b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:40.980516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:40.981128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:40.981406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:40.981672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:40.982334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:40.982635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:40.984046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:40.992750 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 321/679] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Oct 23 00:43:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:40.998548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37f6ec-e372-444f-b85c-3118db2b0c78 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.013861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:41.106288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37f6ec-e372-444f-b85c-3118db2b0c78 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.142700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.143335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.143616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.143889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.154557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.156005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:41.171354 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:41.172414 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:41.173331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:41.173331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:41.173331 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:41.173331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 359/680] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:41.178541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.202274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.272049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:41.312118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a6dfff-70aa-47ba-93a8-eed4c0eff7aa None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.317363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df1d52-0ae7-47a2-9e64-177ab7dbf733 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.332198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:41.354858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.355465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.355747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.356019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.356688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:41.356966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.357096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.357902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.358321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.358581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.359034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.359954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.361638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:41.371401 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:41.372347 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 00:43:41.373238 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:41.375595 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 360/682] 10.222.0.39 () {56 vars in 1132 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:41.381190 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 322/682] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:41.382303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.382754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.487271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af31b39b-b6d2-479d-9f30-c09c29671610 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.490747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d196ca48-6780-4f26-8e01-b67c1e580453 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.522191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:41.524727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc41a40-7797-482c-97a4-909fa266614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.525388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41a40-7797-482c-97a4-909fa266614d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.525662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41a40-7797-482c-97a4-909fa266614d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.525934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc41a40-7797-482c-97a4-909fa266614d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.531633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208e85d3-23a4-46e8-b0f4-2b59ba87f317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.532248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e85d3-23a4-46e8-b0f4-2b59ba87f317 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.532553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e85d3-23a4-46e8-b0f4-2b59ba87f317 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.532825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208e85d3-23a4-46e8-b0f4-2b59ba87f317 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.533144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc41a40-7797-482c-97a4-909fa266614d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.533455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e85d3-23a4-46e8-b0f4-2b59ba87f317 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:41.533754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e85d3-23a4-46e8-b0f4-2b59ba87f317 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.534553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc41a40-7797-482c-97a4-909fa266614d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.535105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208e85d3-23a4-46e8-b0f4-2b59ba87f317 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.540319 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 361/683] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:41.541435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8361-7798-4cbd-a62a-a01ec33e8e4c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.541723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8361-7798-4cbd-a62a-a01ec33e8e4c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.541995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8361-7798-4cbd-a62a-a01ec33e8e4c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.543188 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 362/684] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:41.547946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8361-7798-4cbd-a62a-a01ec33e8e4c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6dc41a40-7797-482c-97a4-909fa266614d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:41.551061 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:41.552001 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 323/685] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245051e-4c6c-4b65-8000-c4b8ae91deb6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.554211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245051e-4c6c-4b65-8000-c4b8ae91deb6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.554908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b245051e-4c6c-4b65-8000-c4b8ae91deb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.572699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b245051e-4c6c-4b65-8000-c4b8ae91deb6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:41.573858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b245051e-4c6c-4b65-8000-c4b8ae91deb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:41.605611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:41.649322 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 324/686] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:41.660891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8361-7798-4cbd-a62a-a01ec33e8e4c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.661275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245051e-4c6c-4b65-8000-c4b8ae91deb6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.697811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.698759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.699266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.699739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.700594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:41.701122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.702745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.712666 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 363/687] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:41.713741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.778544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:41.786560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b245051e-4c6c-4b65-8000-c4b8ae91deb6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.825111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.826059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.826535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.826997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.827999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd22e21-4835-4cfe-a24d-74af8dfb33dc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:41.830916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:41.831429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.833187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.851809 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 325/688] 10.222.0.39 () {56 vars in 1149 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:41.859873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.863344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:41.863954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:41.864231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:41.864544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:41.866255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:41.867761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:41.887729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:41.895960 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 364/689] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:41.926049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:41.989699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63851a1b-745f-49d6-b23e-502555623d6b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.035519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59390ede-1d27-484d-81d1-5d202ee5c721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.036272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59390ede-1d27-484d-81d1-5d202ee5c721 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.036425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59390ede-1d27-484d-81d1-5d202ee5c721 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.036617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59390ede-1d27-484d-81d1-5d202ee5c721 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.049113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc45cb7-5770-425a-9bc0-dbe2a7611fe1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.063645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.090081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbd835c-b166-4966-b901-e044f1d88b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.091146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd835c-b166-4966-b901-e044f1d88b51 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.091146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd835c-b166-4966-b901-e044f1d88b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.091346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbd835c-b166-4966-b901-e044f1d88b51 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.091816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd835c-b166-4966-b901-e044f1d88b51 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.093225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbd835c-b166-4966-b901-e044f1d88b51 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-2dbd835c-b166-4966-b901-e044f1d88b51 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:42.102504 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:42.103629 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:42.104614 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 365/690] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:42.105395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa0bb-a6dc-4400-b090-0150caced96f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.105395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa0bb-a6dc-4400-b090-0150caced96f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.105395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104fa0bb-a6dc-4400-b090-0150caced96f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.106672 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 366/691] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:42 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:42.110306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fa0bb-a6dc-4400-b090-0150caced96f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.126427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59390ede-1d27-484d-81d1-5d202ee5c721 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.127888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59390ede-1d27-484d-81d1-5d202ee5c721 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.151938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.217944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104fa0bb-a6dc-4400-b090-0150caced96f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.240937 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 326/692] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:43:41 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:43:42.246604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0240e6-8a2a-408b-8b48-739584817bc1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.246890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0240e6-8a2a-408b-8b48-739584817bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.247195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0240e6-8a2a-408b-8b48-739584817bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.254575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.254575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.254834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.255018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.255628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:42.255926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.257574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.262535 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 367/693] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:42.263165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.265650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7c0240e6-8a2a-408b-8b48-739584817bc1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:42.267044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7c0240e6-8a2a-408b-8b48-739584817bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:42.307665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.337121 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 327/694] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:42.338787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0240e6-8a2a-408b-8b48-739584817bc1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.387382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227058c7-e2f5-4bef-9989-7910024ad468 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.399856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.428952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.429582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.429857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.430122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.441003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.442418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:42.457103 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:42.458121 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:42.459581 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:42.459581 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:42.459581 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:42.459581 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 368/695] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:42.461418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0240e6-8a2a-408b-8b48-739584817bc1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.465218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.505691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.506978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.507503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.507978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.508835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:42.509326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.511387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.518623 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 328/696] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:42.525263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.571775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.582233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4305da7a-3dcd-4228-a388-cd38a402b9c3 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.620833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.621785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.622282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.622769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.623618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:42.624111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.626048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.638875 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 369/697] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:42.640064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.650570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09672f4-facf-432f-a87b-383993c01c8b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.666094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.687164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.687872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.688222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.688604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.689383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] RBAC: Adding query filter params (interface=public, service_id=da0f2d28708f45b59d7762c59918b68e, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:42.689788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.691225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.696765 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 329/698] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/endpoints?service_id=da0f2d28708f45b59d7762c59918b68e&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:42.701525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.746399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63f40d-cde2-4369-88ae-df2a2dfea749 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.782770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.783396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.783672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.783928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.784531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.785872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:42.795125 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:42.796143 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:42.797015 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 370/699] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:42.797789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.814873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.815943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64b6557-7df6-48e9-946f-f34701939459 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.863318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.864293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.864808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.865456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.874412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.876576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:42.889467 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:42.890343 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:42.891314 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:42.893272 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 330/700] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:42.894044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:42.907054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031089b8-0bb3-4a02-b70d-1d7ec05c215f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:42.940640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:42.941648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:42.942011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:42.942162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:42.942537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:42.943558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:42.945037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:42.951062 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 371/701] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:43:42.951827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:43.022722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bd4a9f-f9d4-4ff5-9ecb-3d8e128d1ad6 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:43.062573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:43.063586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:43.064066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:43.064571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:43.065411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:43.065942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:43.067529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:43.070110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:43.074845 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 331/702] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:43.083427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:43.097959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eafad0-b1e9-43bd-9b77-c5212b8289ca None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:43.142273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:43.142930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:43.143174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:43.143505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:43.144213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:43.144591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:43.146088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:43.163828 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 372/703] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:43.170038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:43.212939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621082b-45d5-4e6c-a201-cba78ec0499b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:43.214683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:43.264264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:43.264941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:43.265434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:43.265706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:43.267265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:43.267571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:43.268950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:43.275952 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 332/704] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:43.281517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:43.302104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c880db19-7dd9-4e87-be05-9be0e47908a4 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:43.342963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:43.343805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:43.344573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:43.344850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:43.345861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] RBAC: Adding query filter params (role.id=eda5bb304e03428dbc184a9019dc3cda, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=232811b7770447b6a12f4a806a7b0119) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:43.346170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:43.348275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:43.360392 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 373/705] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:43:43 2022] GET /identity/v3/role_assignments?user.id=232811b7770447b6a12f4a806a7b0119&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=eda5bb304e03428dbc184a9019dc3cda => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:43.412614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ccf2db-f975-4992-aab6-99eb8bdea089 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:43.451269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd7e506-771d-42ef-aa34-2f791407ea6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:43.451895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7e506-771d-42ef-aa34-2f791407ea6e None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:43.452174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7e506-771d-42ef-aa34-2f791407ea6e None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:43.452483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd7e506-771d-42ef-aa34-2f791407ea6e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:43.453197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7e506-771d-42ef-aa34-2f791407ea6e None admin] RBAC: Adding query filter params (interface=public, service_id=a0c5abc4bc4640199c15d5ecd5469a72, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:43.453511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7e506-771d-42ef-aa34-2f791407ea6e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:43.454899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd7e506-771d-42ef-aa34-2f791407ea6e None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:43.461568 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 333/706] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:43:43 2022] GET /identity/v3/endpoints?service_id=a0c5abc4bc4640199c15d5ecd5469a72&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:43.467500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:43.472559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:43.587584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858fa4ed-ed32-44bd-85d8-ef35e345dbf0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:43.623569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00902c45-e247-4d92-b98e-21112ba9bb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:43.624553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00902c45-e247-4d92-b98e-21112ba9bb24 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:43.625035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00902c45-e247-4d92-b98e-21112ba9bb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:43.625537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00902c45-e247-4d92-b98e-21112ba9bb24 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:43.626338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00902c45-e247-4d92-b98e-21112ba9bb24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:43.627894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00902c45-e247-4d92-b98e-21112ba9bb24 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:43.635151 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 374/707] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:43 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:43:44.178666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9b9fd-792a-4ab2-aad8-25a21c5965e3 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:44.179084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9b9fd-792a-4ab2-aad8-25a21c5965e3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:44.179261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e9b9fd-792a-4ab2-aad8-25a21c5965e3 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:44.180916 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 334/708] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:44.186566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e997b-5766-44e1-add6-a3b0b684a6d1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:44.187029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e997b-5766-44e1-add6-a3b0b684a6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:44.187130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496e997b-5766-44e1-add6-a3b0b684a6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:44.205380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-496e997b-5766-44e1-add6-a3b0b684a6d1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:44.206557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-496e997b-5766-44e1-add6-a3b0b684a6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:44.272249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:44.278660 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 375/709] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:44.554181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9b9fd-792a-4ab2-aad8-25a21c5965e3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:44.586413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:44.661433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e9b9fd-792a-4ab2-aad8-25a21c5965e3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:44.696614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:44.697250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:44.697530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:44.697807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] PATH_INFO: `/v3/services/compute` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:44.698388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:44.699734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:44.708894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:44.709874 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:44.710722 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 335/710] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:44.716275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e997b-5766-44e1-add6-a3b0b684a6d1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:44.821890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496e997b-5766-44e1-add6-a3b0b684a6d1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:44.857521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d5ab3-a0cc-49b2-90bf-0c47a7be3fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:44.858142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5ab3-a0cc-49b2-90bf-0c47a7be3fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:44.858422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5ab3-a0cc-49b2-90bf-0c47a7be3fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:44.858693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d5ab3-a0cc-49b2-90bf-0c47a7be3fd0 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:44.859325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d5ab3-a0cc-49b2-90bf-0c47a7be3fd0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:44.859625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d5ab3-a0cc-49b2-90bf-0c47a7be3fd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:44.861017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d5ab3-a0cc-49b2-90bf-0c47a7be3fd0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:44.866643 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 376/711] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:44.872415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:44.969224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:44.976843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76bc376-2718-41d4-8f3c-90486335fbec None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:45.012946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70813b8f-1116-4314-a6d2-e150def0583f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:45.013552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70813b8f-1116-4314-a6d2-e150def0583f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.013880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70813b8f-1116-4314-a6d2-e150def0583f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.014158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70813b8f-1116-4314-a6d2-e150def0583f None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.014786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70813b8f-1116-4314-a6d2-e150def0583f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:45.015087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70813b8f-1116-4314-a6d2-e150def0583f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:45.016463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70813b8f-1116-4314-a6d2-e150def0583f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:45.021836 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 336/712] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:45.204241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114289-f465-4326-9daa-b10f48dcf991 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.204755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114289-f465-4326-9daa-b10f48dcf991 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.204755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5114289-f465-4326-9daa-b10f48dcf991 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.206344 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 377/713] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:45.212355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bc2f2-a037-43fa-b204-e2eb3ced1169 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.212890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bc2f2-a037-43fa-b204-e2eb3ced1169 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.213363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bc2f2-a037-43fa-b204-e2eb3ced1169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.233583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a19bc2f2-a037-43fa-b204-e2eb3ced1169 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:45.234769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a19bc2f2-a037-43fa-b204-e2eb3ced1169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:45.305102 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 337/714] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:45.577499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114289-f465-4326-9daa-b10f48dcf991 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:45.648751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:45.691963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5114289-f465-4326-9daa-b10f48dcf991 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:45.732153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:45.732833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.733111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.733432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.734021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:45.735445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:45.744907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:45.745909 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:45.746736 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 378/715] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:45.750923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f42b54-f799-4d31-acec-07b4a93f8209 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.750923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f42b54-f799-4d31-acec-07b4a93f8209 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.750923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f42b54-f799-4d31-acec-07b4a93f8209 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.752990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:45.753388 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 338/716] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:45.759013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c029f-9d2a-4e13-9183-0532301db2a3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.759563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c029f-9d2a-4e13-9183-0532301db2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.760056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c029f-9d2a-4e13-9183-0532301db2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.779547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-034c029f-9d2a-4e13-9183-0532301db2a3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:45.781296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-034c029f-9d2a-4e13-9183-0532301db2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:45.859895 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 339/717] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:45.861629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b334ad3-3da9-4ef6-b891-91ce685ece79 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.862117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b334ad3-3da9-4ef6-b891-91ce685ece79 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.862587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b334ad3-3da9-4ef6-b891-91ce685ece79 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.864339 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 340/718] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:45.869362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797cf96-de1b-4ab2-965e-b611fe206500 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.870022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797cf96-de1b-4ab2-965e-b611fe206500 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.870497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a797cf96-de1b-4ab2-965e-b611fe206500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.872087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309bb672-26b1-4708-9c37-ac0e17be57d2 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:45.889675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a797cf96-de1b-4ab2-965e-b611fe206500 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:45.891170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a797cf96-de1b-4ab2-965e-b611fe206500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:45.903295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:45.907647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:45.913851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:45.914461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:45.915150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:45.915422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:45.916052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:45.916504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:45.917998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:45.923626 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 379/719] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:45.929407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:45.986044 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 341/720] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:46.045006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ff9f4-0d46-4cc1-8814-deb2fcb8cb6f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.079789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.080762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.081258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.081750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.082613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:46.083111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.085247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.095670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 380/721] 10.222.0.39 () {56 vars in 1114 bytes} [Sun Oct 23 00:43:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:46.097216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797cf96-de1b-4ab2-965e-b611fe206500 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.134844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.196838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:46.207571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a797cf96-de1b-4ab2-965e-b611fe206500 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.245642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.246577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.247042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.247523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.248265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.249714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.258368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaeabd8-6ac7-49a6-b4e6-6aa61e06376b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.280483 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 342/722] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:43:46.295473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.299283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:46.307834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.308528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.308806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.309073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] PATH_INFO: `/v3/services/image` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.309663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.311539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:46.322636 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:46.323729 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:46.324609 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:46.327013 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 381/723] 10.222.0.39 () {56 vars in 1105 bytes} [Sun Oct 23 00:43:46 2022] GET /identity/v3/services/image => generated 86 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:46.340426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.445145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7cd400-fd66-4fad-ae83-d05b93706496 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.447709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc084342-7603-4710-8f01-f377c93a2825 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.464166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:46.486419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.487054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.487332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.487454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.487607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.487936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.488169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.488261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.488502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.489126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] RBAC: Adding query filter params (name=image) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:46.489436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.491017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.491443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 382/724] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:46.500883 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:46.501896 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:46.503363 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:46.504127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.504199 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 343/725] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:46.508537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.597362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:46.614113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc07c97-549b-4732-be06-a4610e0ec344 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.647332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2133b8de-f3d7-45ef-aa44-89bfc322b91d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.649704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.650321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.650598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.650869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.652083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] RBAC: Adding query filter params (type=image) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:46.652083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.653178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.658209 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 383/726] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:46 2022] GET /identity/v3/services?type=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:46.690665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.691731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.692275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.692806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.693737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:46.694248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.695954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.702396 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 344/727] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:46.712351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.762839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:46.832389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ef6a4-ce7d-4909-bae9-0da7e6b2cc3b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:46.869954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d56eaa-abd7-4e1f-a41e-7167b6d87c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:46.870483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d56eaa-abd7-4e1f-a41e-7167b6d87c11 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:46.870762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d56eaa-abd7-4e1f-a41e-7167b6d87c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:46.871035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d56eaa-abd7-4e1f-a41e-7167b6d87c11 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:46.871676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d56eaa-abd7-4e1f-a41e-7167b6d87c11 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:46.871967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d56eaa-abd7-4e1f-a41e-7167b6d87c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:46.873377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d56eaa-abd7-4e1f-a41e-7167b6d87c11 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:46.879168 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 384/728] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:46.885069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:46.990152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e5792-641a-43d8-b11a-1c20914a152c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:47.021553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:47.026937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5225272a-fbd8-4acc-938d-81efef9110ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:47.027552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225272a-fbd8-4acc-938d-81efef9110ec None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:47.028504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225272a-fbd8-4acc-938d-81efef9110ec None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:47.028504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5225272a-fbd8-4acc-938d-81efef9110ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:47.028775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225272a-fbd8-4acc-938d-81efef9110ec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:47.030121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5225272a-fbd8-4acc-938d-81efef9110ec None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:47.056464 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 345/729] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:46 2022] POST /identity/v3/endpoints => generated 349 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:43:47.285432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f7949-6b42-49b7-be8d-59fcec71e5ed None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:47.285948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f7949-6b42-49b7-be8d-59fcec71e5ed None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:47.285948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f7949-6b42-49b7-be8d-59fcec71e5ed None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:47.288057 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 385/730] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:47.293522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c631d-95fc-401a-a265-ead95993497f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:47.293803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c631d-95fc-401a-a265-ead95993497f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:47.294074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c631d-95fc-401a-a265-ead95993497f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:47.313321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-456c631d-95fc-401a-a265-ead95993497f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:47.314520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-456c631d-95fc-401a-a265-ead95993497f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:47.393756 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 346/731] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:47.665516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f7949-6b42-49b7-be8d-59fcec71e5ed None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:47.772550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f7949-6b42-49b7-be8d-59fcec71e5ed None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:47.780026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:47.814116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b512843-a4c1-49e2-9745-340bad068a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:47.815078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b512843-a4c1-49e2-9745-340bad068a58 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:47.815559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b512843-a4c1-49e2-9745-340bad068a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:47.816079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b512843-a4c1-49e2-9745-340bad068a58 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:47.816963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b512843-a4c1-49e2-9745-340bad068a58 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:47.818563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b512843-a4c1-49e2-9745-340bad068a58 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:47.841113 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 386/732] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:47 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:48.632120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6c08-1aed-4b8b-ac90-a9b9cdbd646b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:48.633195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6c08-1aed-4b8b-ac90-a9b9cdbd646b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:48.633676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33f6c08-1aed-4b8b-ac90-a9b9cdbd646b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:48.635452 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 347/733] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:48.641337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d098-78a2-4552-b875-ba9127bc12e9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:48.642155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d098-78a2-4552-b875-ba9127bc12e9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:48.642665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d098-78a2-4552-b875-ba9127bc12e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:48.661065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7cb4d098-78a2-4552-b875-ba9127bc12e9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:48.662696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7cb4d098-78a2-4552-b875-ba9127bc12e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:48.742571 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 387/734] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:48.892515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d4518-b6d0-4822-9bba-09bb3e40b417 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:48.892870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d4518-b6d0-4822-9bba-09bb3e40b417 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:48.893237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d4518-b6d0-4822-9bba-09bb3e40b417 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:48.894959 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 348/735] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:48.900392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52df4a6-96ed-4585-9623-b2b54ce8ffa6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:48.900806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52df4a6-96ed-4585-9623-b2b54ce8ffa6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:48.901176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52df4a6-96ed-4585-9623-b2b54ce8ffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:48.919576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a52df4a6-96ed-4585-9623-b2b54ce8ffa6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:48.920774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a52df4a6-96ed-4585-9623-b2b54ce8ffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:48.992400 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 388/736] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:49.040335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d4518-b6d0-4822-9bba-09bb3e40b417 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:49.146459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d4518-b6d0-4822-9bba-09bb3e40b417 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:49.163874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:49.187984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:49.188650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:49.188983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:49.189256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:49.189913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:49.191712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:49.194765 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING keystone.server.flask.common [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 23 00:43:49.214047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.sql.core [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 00:43:49.214047 dv-cinder-master-3-smb devstack@keystone.service[81371]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 00:43:49.214047 dv-cinder-master-3-smb devstack@keystone.service[81371]: [parameters: {'id': 'bebd8aa2a48e461bad6f5acaeb11c737', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 23 00:43:49.214047 dv-cinder-master-3-smb devstack@keystone.service[81371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-0d34f781-9f96-4bd7-9ea7-7cbb302f70ce None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bebd8aa2a48e461bad6f5acaeb11c737.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bebd8aa2a48e461bad6f5acaeb11c737. Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:49.231278 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 00:43:49.232077 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application yield resource Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise self.value Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application session.commit() Oct 23 00:43:49.233058 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.flush() Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application compat.raise_( Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 00:43:49.233974 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 00:43:49.235713 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:49.237802 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [parameters: {'id': 'bebd8aa2a48e461bad6f5acaeb11c737', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:49.240134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:49.242067 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bebd8aa2a48e461bad6f5acaeb11c737. Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.249848 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 349/737] 10.222.0.39 () {60 vars in 1140 bytes} [Sun Oct 23 00:43:49 2022] POST /identity/v3/projects => generated 262 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:49.251159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52df4a6-96ed-4585-9623-b2b54ce8ffa6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:49.279010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a029d8-f611-45d9-b15f-1b178afef759 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:49.279294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a029d8-f611-45d9-b15f-1b178afef759 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:49.279549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a029d8-f611-45d9-b15f-1b178afef759 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:49.283929 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 350/738] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:49.289502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0841be8-66c7-4ea8-9b06-7f06cbc332fc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:49.289782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0841be8-66c7-4ea8-9b06-7f06cbc332fc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:49.290054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0841be8-66c7-4ea8-9b06-7f06cbc332fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:49.294064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:49.310056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d0841be8-66c7-4ea8-9b06-7f06cbc332fc None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:49.311937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d0841be8-66c7-4ea8-9b06-7f06cbc332fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:49.345044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52df4a6-96ed-4585-9623-b2b54ce8ffa6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:49.379148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:49.379774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:49.380052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:49.380401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:49.382304 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 351/739] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:49.382944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0841be8-66c7-4ea8-9b06-7f06cbc332fc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:49.390257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:49.391955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:49.404969 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:49.405882 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:49.406759 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:49.409241 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 389/740] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:49.415523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:49.516026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:49.523318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0841be8-66c7-4ea8-9b06-7f06cbc332fc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:49.526781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:49.560132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119764f3-ad42-42c0-a3e4-ab83f62fd4ee None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:49.597679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:49.598726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:49.601749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:49.602173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:49.603072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:49.604725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:49.629019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:49.632378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:49.632894 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 352/741] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:49 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:49.633754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:49.634215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:49.635032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:49.635517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:49.639033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:49.657847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:49.672241 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 390/742] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:49.895538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e75b845-5a9b-4099-b713-5adeb74fa3b0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:49.938800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d85d4f-aca3-44ec-b6f9-e1a35730541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:49.939361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85d4f-aca3-44ec-b6f9-e1a35730541c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:49.939641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85d4f-aca3-44ec-b6f9-e1a35730541c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:49.939906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d85d4f-aca3-44ec-b6f9-e1a35730541c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:49.941206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d85d4f-aca3-44ec-b6f9-e1a35730541c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:49.941882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d85d4f-aca3-44ec-b6f9-e1a35730541c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-81d85d4f-aca3-44ec-b6f9-e1a35730541c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:49.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:49.955332 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:49.956171 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 353/743] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:49.966131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:50.085240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ba65f0-2794-40ae-b8f2-54807017f3e0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:50.085908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808a5c-51c5-40ee-a693-b72b1a954ace None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.086198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808a5c-51c5-40ee-a693-b72b1a954ace None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.086474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76808a5c-51c5-40ee-a693-b72b1a954ace None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.088436 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 354/744] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:50.093112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c798118-4303-493e-9dd2-1aba4c8acc4c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.093399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c798118-4303-493e-9dd2-1aba4c8acc4c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.093674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c798118-4303-493e-9dd2-1aba4c8acc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.111672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8c798118-4303-493e-9dd2-1aba4c8acc4c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:50.113111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8c798118-4303-493e-9dd2-1aba4c8acc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:50.126884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:50.128471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.128975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.129458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.131053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:50.132164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:50.133083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:50.135063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:50.142422 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 391/745] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:50.147253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:50.158677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:50.200700 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 355/746] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:50 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:50.264982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3987fd8-da0f-43be-b9a0-e40ab63af6ec None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:50.300203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:50.301131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.301588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.302631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.303435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:50.303915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:50.305463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:50.310832 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 392/747] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:50.319419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c798118-4303-493e-9dd2-1aba4c8acc4c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:50.418219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c798118-4303-493e-9dd2-1aba4c8acc4c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:50.452130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:50.452707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.452985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.453255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.453958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] RBAC: Adding query filter params (interface=admin, service_id=a0c5abc4bc4640199c15d5ecd5469a72, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:50.454266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:50.455616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:50.461424 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 356/748] 10.222.0.39 () {56 vars in 1249 bytes} [Sun Oct 23 00:43:50 2022] GET /identity/v3/endpoints?service_id=a0c5abc4bc4640199c15d5ecd5469a72&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:50.466889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:50.491664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:50.510061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:50.590873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d263d29-ff6c-4dcb-85a8-d3e02b7f7992 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:50.604637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3afdff-bdb6-48cb-9766-2e19493ddd08 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:50.613009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:50.636421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:50.637370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.637821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.638384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.639433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:50.641317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:50.643885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:50.644728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.645102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.645523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.646208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:50.647640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:50.649926 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 393/749] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:50 2022] GET /identity/v3/services => generated 1957 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:50.660080 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:50.662029 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:50.663062 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:50.665596 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 357/750] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:50 2022] GET /identity/v3/services/compute => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:50.674077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:50.789964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f628bb72-c61b-433a-a4d1-4cfd5fe05ba5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:50.836108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:50.836781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.837065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.837480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.838198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:50.838509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:50.839939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:50.841217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:50.846270 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 394/751] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:50.853370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:50.937030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:50.960367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825051f1-b5b9-46c2-99fb-67602934dd33 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:50.995105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:50.995768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:50.996046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:50.996342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:50.997003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:50.997308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:50.998659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:51.003884 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 358/752] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:51.010140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:51.117775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d56eb-8c09-406d-9799-b9ba070e1a74 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:51.154514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b966eb-9b22-4361-bd32-06537e8fd799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:51.155183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b966eb-9b22-4361-bd32-06537e8fd799 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:51.155467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b966eb-9b22-4361-bd32-06537e8fd799 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:51.155741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b966eb-9b22-4361-bd32-06537e8fd799 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:51.156473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b966eb-9b22-4361-bd32-06537e8fd799 None admin] RBAC: Adding query filter params (interface=public, service_id=5d967b237c844d658f21bc5ce73f22dd, region_id=RegionOne) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:51.156780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b966eb-9b22-4361-bd32-06537e8fd799 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:51.158137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b966eb-9b22-4361-bd32-06537e8fd799 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:51.163867 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 395/753] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:43:51 2022] GET /identity/v3/endpoints?service_id=5d967b237c844d658f21bc5ce73f22dd&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:51.169589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:51.277136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df98de51-5e9d-483d-b580-70b53a49a789 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:51.311482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce001d66-2b1a-48e1-b0c7-4114cba4f546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:51.312093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce001d66-2b1a-48e1-b0c7-4114cba4f546 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:51.312400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce001d66-2b1a-48e1-b0c7-4114cba4f546 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:51.312676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce001d66-2b1a-48e1-b0c7-4114cba4f546 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:51.313277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce001d66-2b1a-48e1-b0c7-4114cba4f546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:51.314697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce001d66-2b1a-48e1-b0c7-4114cba4f546 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:51.322066 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 359/754] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:51 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:43:52.050578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe867-95ee-456f-802a-92701083b09e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.051594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe867-95ee-456f-802a-92701083b09e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.052114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe867-95ee-456f-802a-92701083b09e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.053882 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 396/755] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:52.059616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f061bd-c36e-4e62-b2cd-bdbab2e7b3d7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.060483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f061bd-c36e-4e62-b2cd-bdbab2e7b3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.060980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f061bd-c36e-4e62-b2cd-bdbab2e7b3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.074593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:52.080946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-04f061bd-c36e-4e62-b2cd-bdbab2e7b3d7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:52.082357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-04f061bd-c36e-4e62-b2cd-bdbab2e7b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:52.126401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b8367-39b0-4a79-befb-ba9c6368a6b0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.126733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b8367-39b0-4a79-befb-ba9c6368a6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.126958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b8367-39b0-4a79-befb-ba9c6368a6b0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.132023 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 397/756] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:52.134289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50823c0-f54c-4a7d-92e2-a9f60aaeb3bd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.134929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50823c0-f54c-4a7d-92e2-a9f60aaeb3bd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.135464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50823c0-f54c-4a7d-92e2-a9f60aaeb3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.159481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e50823c0-f54c-4a7d-92e2-a9f60aaeb3bd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:52.161010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e50823c0-f54c-4a7d-92e2-a9f60aaeb3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:52.163842 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 360/757] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:52 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:52.222259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:52.233296 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 398/758] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:52 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:52.437518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f061bd-c36e-4e62-b2cd-bdbab2e7b3d7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:52.516125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50823c0-f54c-4a7d-92e2-a9f60aaeb3bd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:52.545154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f061bd-c36e-4e62-b2cd-bdbab2e7b3d7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:52.589160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:52.590074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.590352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.590622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] PATH_INFO: `/v3/services/image` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.591373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:52.593316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:52.596220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:52.604970 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:52.606708 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:52.607571 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 361/759] 10.222.0.39 () {56 vars in 1105 bytes} [Sun Oct 23 00:43:52 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:52.617473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:52.631297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50823c0-f54c-4a7d-92e2-a9f60aaeb3bd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:52.676616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:52.677828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.678132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.678906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.679565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:52.681293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:52.685060 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING keystone.server.flask.common [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 23 00:43:52.703910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.sql.core [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 00:43:52.703910 dv-cinder-master-3-smb devstack@keystone.service[81372]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 23 00:43:52.703910 dv-cinder-master-3-smb devstack@keystone.service[81372]: [parameters: {'user_id': '11524d48a12b40f08b224d501c3385bf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 23 00:43:52.703910 dv-cinder-master-3-smb devstack@keystone.service[81372]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81372) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:52.713779 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:52.714654 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application yield resource Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise self.value Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application session.commit() Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 00:43:52.715619 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.session.flush() Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application compat.raise_( Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 00:43:52.716517 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:52.723651 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:52.726203 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [parameters: {'user_id': '11524d48a12b40f08b224d501c3385bf', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:52.728175 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 23 00:43:52.730174 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 399/760] 10.222.0.39 () {60 vars in 1131 bytes} [Sun Oct 23 00:43:52 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:52.732496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:52.742049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74ce7f-6caf-4530-8c50-63fdb7557959 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:52.778901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:52.779850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.780346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.780831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.781641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] RBAC: Adding query filter params (name=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:52.782131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:52.783887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:52.794828 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 362/761] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:52.796079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:52.803334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:52.864003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef4782-f3d3-4eb5-8c36-0a44507e18f8 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:52.889015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:52.900851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b2a54a-b599-4458-9bfe-a93eb41ad481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:52.901494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a54a-b599-4458-9bfe-a93eb41ad481 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.901790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a54a-b599-4458-9bfe-a93eb41ad481 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.902788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b2a54a-b599-4458-9bfe-a93eb41ad481 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.910160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2a54a-b599-4458-9bfe-a93eb41ad481 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:52.912123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b2a54a-b599-4458-9bfe-a93eb41ad481 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-35b2a54a-b599-4458-9bfe-a93eb41ad481 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:52.924702 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:52.925850 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 00:43:52.926799 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:52.931710 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 400/762] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:43:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:52.933377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28936-5209-4103-9ad4-be4ac872bea7 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.933661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28936-5209-4103-9ad4-be4ac872bea7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.933942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c28936-5209-4103-9ad4-be4ac872bea7 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.938151 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 401/763] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:52.939537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560d600-1797-4a7e-a2fc-b309bf289874 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:52.941455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28936-5209-4103-9ad4-be4ac872bea7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:52.976274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a47efa-529a-4686-9a8d-5242f1cff7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:52.977157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47efa-529a-4686-9a8d-5242f1cff7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.977431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47efa-529a-4686-9a8d-5242f1cff7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.977699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a47efa-529a-4686-9a8d-5242f1cff7b4 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:52.978315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a47efa-529a-4686-9a8d-5242f1cff7b4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:52.978610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a47efa-529a-4686-9a8d-5242f1cff7b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:52.979953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a47efa-529a-4686-9a8d-5242f1cff7b4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:52.985124 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 363/764] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:52.991423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeed4-9114-404d-84b3-8fdcc6746e81 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:52.991699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeed4-9114-404d-84b3-8fdcc6746e81 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:52.991962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daeed4-9114-404d-84b3-8fdcc6746e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.010666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-03daeed4-9114-404d-84b3-8fdcc6746e81 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:53.011848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-03daeed4-9114-404d-84b3-8fdcc6746e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:53.047650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c28936-5209-4103-9ad4-be4ac872bea7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:53.063833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:53.085311 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 364/765] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:52 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:53.086763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daeed4-9114-404d-84b3-8fdcc6746e81 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:53.089197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:53.089856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.090133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.090403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.091096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:53.091474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:53.092945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:53.127973 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 402/766] 10.222.0.39 () {56 vars in 1101 bytes} [Sun Oct 23 00:43:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:53.158276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:53.207812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daeed4-9114-404d-84b3-8fdcc6746e81 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:53.248146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:53.249044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.249469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.249881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.250799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] RBAC: Adding query filter params (interface=public, service_id=c2eac6053f804dffac007113c38f046b, region_id=RegionOne) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:53.251269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:53.252910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:53.260418 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 365/767] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:43:53 2022] GET /identity/v3/endpoints?service_id=c2eac6053f804dffac007113c38f046b&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:53.268378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:53.363075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:53.378261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa8bbb-184f-4087-98d1-c60d798e1fc0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:53.416641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:53.417255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.417531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.417802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.418400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:53.419759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:53.426708 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 403/768] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:43:53 2022] GET /identity/v3/services => generated 1957 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:43:53.492614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c469a735-c674-4e9c-acc7-363e499a1d04 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:53.518605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:53.531686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c084e-4406-48f5-aad2-d22c343dfd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:53.532358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c084e-4406-48f5-aad2-d22c343dfd82 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.532635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c084e-4406-48f5-aad2-d22c343dfd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.532903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c084e-4406-48f5-aad2-d22c343dfd82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.533515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c084e-4406-48f5-aad2-d22c343dfd82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:53.534898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c084e-4406-48f5-aad2-d22c343dfd82 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-c93c084e-4406-48f5-aad2-d22c343dfd82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:53.545223 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:53.546501 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 00:43:53.547649 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:53.551372 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 366/769] 10.222.0.39 () {56 vars in 1126 bytes} [Sun Oct 23 00:43:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:53.553411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:53.568337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d85290-07e1-404a-80b5-0887d55837cd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.569835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d85290-07e1-404a-80b5-0887d55837cd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.569835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d85290-07e1-404a-80b5-0887d55837cd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.571501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 367/770] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:53.577631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec7e6b-9074-4538-8444-b90b76897c8f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.577910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec7e6b-9074-4538-8444-b90b76897c8f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.578313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec7e6b-9074-4538-8444-b90b76897c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.591994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:53.600553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a8ec7e6b-9074-4538-8444-b90b76897c8f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:53.601776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a8ec7e6b-9074-4538-8444-b90b76897c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:53.665642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73c217a-fe92-4bd1-91d6-a07c91889570 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:53.683945 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 368/771] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:53.702323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:53.702923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.703195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.703459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.704077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:53.704408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:53.705765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:53.715859 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 404/772] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:53.717084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec7e6b-9074-4538-8444-b90b76897c8f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:53.819487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec7e6b-9074-4538-8444-b90b76897c8f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:53.837680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:53.856519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:53.857130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:53.857406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:53.857674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:53.858358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:53.858666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:53.860012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:53.869573 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 369/773] 10.222.0.39 () {56 vars in 1122 bytes} [Sun Oct 23 00:43:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:53.870786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:53.965947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:53.970130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:53.973522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc7c6d-a0c1-47dc-bf1e-9b9cbf199535 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:54.010760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:54.011792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:54.012083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:54.012390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:54.013090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:54.015141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:54.040342 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 405/774] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:43:54.086186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a492844-0c22-4976-8156-b117885e7ee1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:54.129906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:54.131078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:54.131350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:54.131616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] PATH_INFO: `/v3/services/compute` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:54.132465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:54.133811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:54.143150 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:54.144219 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:54.145115 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 370/775] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:54.150845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:54.257790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230358d0-79cb-40d7-b75c-91161aff3653 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:54.293582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:54.294432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:54.294862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:54.295285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:54.296083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:54.296559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:54.298064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:54.303073 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 406/776] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:54.308996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:54.354352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:54.413475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a876ba-1178-4552-b70a-ce48fee89aef None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:54.447891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6f1a9-1a7f-4e1d-8543-52470cd0637a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:54.448790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6f1a9-1a7f-4e1d-8543-52470cd0637a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:54.449277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6f1a9-1a7f-4e1d-8543-52470cd0637a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:54.449745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6f1a9-1a7f-4e1d-8543-52470cd0637a None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:54.450544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6f1a9-1a7f-4e1d-8543-52470cd0637a None admin] RBAC: Adding query filter params (type=compute) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:54.451025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6f1a9-1a7f-4e1d-8543-52470cd0637a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:54.452580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6f1a9-1a7f-4e1d-8543-52470cd0637a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:54.457507 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 371/777] 10.222.0.39 () {56 vars in 1112 bytes} [Sun Oct 23 00:43:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:54.463266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:54.490195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:54.682687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ffa85-b1ee-4140-a83e-e91e2cd9344f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:54.717107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc295c-5133-40df-8aa9-5c83cbc0c44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:54.717715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc295c-5133-40df-8aa9-5c83cbc0c44e None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:54.717994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc295c-5133-40df-8aa9-5c83cbc0c44e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:54.718267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc295c-5133-40df-8aa9-5c83cbc0c44e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:54.718941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc295c-5133-40df-8aa9-5c83cbc0c44e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:54.720289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc295c-5133-40df-8aa9-5c83cbc0c44e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:54.740330 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 407/778] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:55.612149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4942d-c7f3-4446-a2bd-5761f36aa37e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:55.612719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4942d-c7f3-4446-a2bd-5761f36aa37e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:55.612719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4942d-c7f3-4446-a2bd-5761f36aa37e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:55.614356 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 372/779] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:55.619819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e7a2b-a9c4-4d23-b0a6-7afda6358000 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:55.620232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e7a2b-a9c4-4d23-b0a6-7afda6358000 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:55.620442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e7a2b-a9c4-4d23-b0a6-7afda6358000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:55.639195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ca3e7a2b-a9c4-4d23-b0a6-7afda6358000 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:55.640774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ca3e7a2b-a9c4-4d23-b0a6-7afda6358000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:55.717047 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 408/780] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:55.981834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4942d-c7f3-4446-a2bd-5761f36aa37e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:56.090905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4942d-c7f3-4446-a2bd-5761f36aa37e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:56.094292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:56.130922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f112fee-3a23-4957-b74b-53781149fab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:56.131554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f112fee-3a23-4957-b74b-53781149fab9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.132078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f112fee-3a23-4957-b74b-53781149fab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.132367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f112fee-3a23-4957-b74b-53781149fab9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.132941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f112fee-3a23-4957-b74b-53781149fab9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:56.134273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f112fee-3a23-4957-b74b-53781149fab9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-8f112fee-3a23-4957-b74b-53781149fab9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:56.143026 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:56.143989 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:56.145290 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 373/781] 10.222.0.39 () {56 vars in 1105 bytes} [Sun Oct 23 00:43:55 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:56.150697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e7a2b-a9c4-4d23-b0a6-7afda6358000 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:56.159353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:56.206012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82ac9-cbb6-4f8b-a88d-5e7cc0579483 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.206289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82ac9-cbb6-4f8b-a88d-5e7cc0579483 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.206555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa82ac9-cbb6-4f8b-a88d-5e7cc0579483 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.208027 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 374/782] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:56.213214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb9d6-3bb0-4bc1-ab7d-46cc66d9fae2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.213493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb9d6-3bb0-4bc1-ab7d-46cc66d9fae2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.213766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6eb9d6-3bb0-4bc1-ab7d-46cc66d9fae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.231952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-af6eb9d6-3bb0-4bc1-ab7d-46cc66d9fae2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:56.233088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-af6eb9d6-3bb0-4bc1-ab7d-46cc66d9fae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:56.254825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e7a2b-a9c4-4d23-b0a6-7afda6358000 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:56.289073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:56.289673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.289949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.290222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.290839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] RBAC: Adding query filter params (name=image) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:56.291205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:56.292653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:56.297380 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 409/783] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:56.299779 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 375/784] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:56.305305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:56.371421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:56.402982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318d5958-eaf4-430a-b4fe-4177b0432412 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:56.435544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:56.436145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.436448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.436714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.437336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] RBAC: Adding query filter params (type=image) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:56.437625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:56.438956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:56.444325 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 410/785] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:56.449934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb9d6-3bb0-4bc1-ab7d-46cc66d9fae2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:56.544124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:56.548562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6eb9d6-3bb0-4bc1-ab7d-46cc66d9fae2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:56.567008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:56.585509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:56.586483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.586767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.587033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.587843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:56.589220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:56.609899 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 376/786] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:43:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:56.688144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd94970-8319-492f-91f9-d0868e9a2100 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:56.725304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:56.733145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf244d-a9dc-466b-a221-f1a818c3c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:56.734306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf244d-a9dc-466b-a221-f1a818c3c26d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.734586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf244d-a9dc-466b-a221-f1a818c3c26d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.734855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf244d-a9dc-466b-a221-f1a818c3c26d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.736101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf244d-a9dc-466b-a221-f1a818c3c26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:56.737802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf244d-a9dc-466b-a221-f1a818c3c26d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:56.740285 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 411/787] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:56.745941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:56.859663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5c5b1-c5b8-47f8-ab3a-f1c1b15f94b4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:56.899591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea004899-b960-4ef5-9d85-14b0b8ac7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:56.900199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea004899-b960-4ef5-9d85-14b0b8ac7d59 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.900495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea004899-b960-4ef5-9d85-14b0b8ac7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.900764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea004899-b960-4ef5-9d85-14b0b8ac7d59 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.901417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea004899-b960-4ef5-9d85-14b0b8ac7d59 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:56.902827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea004899-b960-4ef5-9d85-14b0b8ac7d59 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:56.936340 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 377/788] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:43:56 2022] POST /identity/v3/projects => generated 311 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:43:56.985660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeed1c-3ceb-46fe-bf1b-6af752cb04c1 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.985935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeed1c-3ceb-46fe-bf1b-6af752cb04c1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.986208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeed1c-3ceb-46fe-bf1b-6af752cb04c1 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:56.987623 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 412/789] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:56.993629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddd3db-245d-421e-a220-12560bfc1a82 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:56.993939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddd3db-245d-421e-a220-12560bfc1a82 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:56.994180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddd3db-245d-421e-a220-12560bfc1a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:57.007300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:57.013934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f4ddd3db-245d-421e-a220-12560bfc1a82 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:57.015995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f4ddd3db-245d-421e-a220-12560bfc1a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:57.089764 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 378/790] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:57.352676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeed1c-3ceb-46fe-bf1b-6af752cb04c1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:57.456206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeed1c-3ceb-46fe-bf1b-6af752cb04c1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:57.491463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:57.492100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:57.492404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:57.492680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:57.502635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:43:57.508348 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:57.509293 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:57.510559 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 413/791] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Oct 23 00:43:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:57.515333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddd3db-245d-421e-a220-12560bfc1a82 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:57.618841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ddd3db-245d-421e-a220-12560bfc1a82 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:57.653130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:57.653740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:57.654017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:57.654286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:57.654944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:57.655243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:57.656629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:57.658857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:57.670420 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 379/792] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Oct 23 00:43:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:57.671598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:57.730171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:57.772679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec196ec0-119f-4b95-a877-7aabf3b8b001 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:57.805496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:57.806120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:57.806446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:57.806722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:57.821510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:57.823026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:57.838069 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:57.839008 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:57.839889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:57.839889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:57.839889 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:57.839889 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 414/793] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:57.845874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:57.949366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8415e-8199-4322-b068-fd6f137220ba None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:57.982038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:57.982652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:57.982930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:57.983201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:57.983833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:57.984132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:57.985562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:58.002512 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 380/794] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:58.003732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:58.103476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3ba01-9199-4197-9069-16f3a514d785 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:58.113434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:58.138916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:58.139541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.139819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.140091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.148074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:58.149535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:43:58.160463 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:58.161510 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:58.162465 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 415/795] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Oct 23 00:43:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:58.167231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:58.227507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:58.269705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d01e7-512d-4c6c-a46d-7c4da56d5fec None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:58.303421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:58.304022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.304295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.304584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.305298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:58.305612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:58.307007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:58.323966 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 381/796] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Oct 23 00:43:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:58.329522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:58.428843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931e720-3346-449b-a6c9-774a0381b9cd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:58.462001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:58.462631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.462904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.463167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.473490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:58.476453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:58.495291 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:58.496244 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:58.497440 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:58.497440 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:58.497440 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:58.497981 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 416/797] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:58.502139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:58.601706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54680c-b12a-4de4-aa4b-929164b378e1 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:58.618595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:58.637716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ae414-e99b-4fa8-96e8-c28902d9f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:58.638403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ae414-e99b-4fa8-96e8-c28902d9f240 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.638745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ae414-e99b-4fa8-96e8-c28902d9f240 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.639096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ae414-e99b-4fa8-96e8-c28902d9f240 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.639768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ae414-e99b-4fa8-96e8-c28902d9f240 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:58.640128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ae414-e99b-4fa8-96e8-c28902d9f240 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:58.641553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ae414-e99b-4fa8-96e8-c28902d9f240 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:58.655344 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 382/798] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:58.659389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:58.684992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:58.766451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe03a23-6409-4537-9993-a1b1e021d5fc None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:58.799986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:58.800641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.800923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.801192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.812742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:58.812742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:58.819641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042c787-e784-430e-bc09-f238586c2479 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.819928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042c787-e784-430e-bc09-f238586c2479 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.820202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0042c787-e784-430e-bc09-f238586c2479 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.821712 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 383/799] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:43:58.826513 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:58.827404 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:43:58.828333 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:58.828784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fba3a4-dd04-4628-a434-b577ddc7bc8c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.828866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 417/800] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:43:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:58.829759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fba3a4-dd04-4628-a434-b577ddc7bc8c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.830040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fba3a4-dd04-4628-a434-b577ddc7bc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.834011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:58.849180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f5fba3a4-dd04-4628-a434-b577ddc7bc8c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:58.850541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f5fba3a4-dd04-4628-a434-b577ddc7bc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:58.920631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:58.923844 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 384/801] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:43:58.940315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ba3423-ef60-4550-a2b7-82dad4f00e6f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:58.974253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc5d93-47ac-445b-bcd6-5ed50667f201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:58.974862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc5d93-47ac-445b-bcd6-5ed50667f201 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:58.975141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc5d93-47ac-445b-bcd6-5ed50667f201 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:58.975417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc5d93-47ac-445b-bcd6-5ed50667f201 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:58.976112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc5d93-47ac-445b-bcd6-5ed50667f201 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:58.976449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc5d93-47ac-445b-bcd6-5ed50667f201 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:58.977863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc5d93-47ac-445b-bcd6-5ed50667f201 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:58.990510 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 418/802] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:43:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:58.996294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fba3a4-dd04-4628-a434-b577ddc7bc8c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:59.106396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fba3a4-dd04-4628-a434-b577ddc7bc8c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:59.113701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:59.142831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:59.143497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.143778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.144050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/f04fb9e1d1bd42ac815078b61f6a17f8/roles/eda5bb304e03428dbc184a9019dc3cda` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.146691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=f04fb9e1d1bd42ac815078b61f6a17f8, role_id=eda5bb304e03428dbc184a9019dc3cda)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:59.148191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:59.149607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc507-4605-4462-960b-a4dec3134875 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.149894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc507-4605-4462-960b-a4dec3134875 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.150173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3bc507-4605-4462-960b-a4dec3134875 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.151496 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 419/803] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:43:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:43:59.157084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11ea0-d509-463c-86de-6fa5ecfc2ebd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.157367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11ea0-d509-463c-86de-6fa5ecfc2ebd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.157640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a11ea0-d509-463c-86de-6fa5ecfc2ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.164123 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 385/804] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:43:58 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/f04fb9e1d1bd42ac815078b61f6a17f8/roles/eda5bb304e03428dbc184a9019dc3cda => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:43:59.177235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-24a11ea0-d509-463c-86de-6fa5ecfc2ebd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:43:59.178455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-24a11ea0-d509-463c-86de-6fa5ecfc2ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:43:59.202934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:59.255569 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 420/805] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:43:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:43:59.315209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaeb1b3-7923-4497-941d-413849a7080a None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:59.352995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:59.353609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.353895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.354176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.354827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:59.356179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:59.364660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.sql.core [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 00:43:59.364660 dv-cinder-master-3-smb devstack@keystone.service[81371]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 00:43:59.364660 dv-cinder-master-3-smb devstack@keystone.service[81371]: [parameters: {'id': 'd22aeda8e6094eaf9f8904b08c46c2ba', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 00:43:59.364660 dv-cinder-master-3-smb devstack@keystone.service[81371]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81371) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:59.371795 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 00:43:59.372872 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application yield resource Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application next(self.gen) Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise self.value Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application session.commit() Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 00:43:59.374389 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.session.flush() Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application compat.raise_( Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 00:43:59.375528 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 00:43:59.376631 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application conn.query(q) Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application result.read() Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 00:43:59.377421 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [parameters: {'id': 'd22aeda8e6094eaf9f8904b08c46c2ba', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 00:43:59.378401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:59.379230 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:43:59.380909 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 386/806] 10.222.0.39 () {60 vars in 1137 bytes} [Sun Oct 23 00:43:59 2022] POST /identity/v3/domains => generated 190 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:43:59.381991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11ea0-d509-463c-86de-6fa5ecfc2ebd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:59.450549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:59.482998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a11ea0-d509-463c-86de-6fa5ecfc2ebd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:59.517810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:59.518443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.518722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.518995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.529570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:59.531012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:59.537159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:43:59.545846 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:43:59.547029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:43:59.548273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:43:59.548273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:43:59.548273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:43:59.551783 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 421/807] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:43:59.552689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:59.572007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:59.651692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a639738b-83f8-47f2-9efa-57e862a94106 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:59.657304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df792f73-276c-485a-8566-3569936a41f5 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:59.686059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:59.687558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:59.688185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.688615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.688818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.689911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:59.691320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:59.698705 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 387/808] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:43:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:43:59.699256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b26ce-bee8-4b0a-ab69-0b490f98cda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:59.699528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:43:59.699876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b26ce-bee8-4b0a-ab69-0b490f98cda4 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.700162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b26ce-bee8-4b0a-ab69-0b490f98cda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.700471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b26ce-bee8-4b0a-ab69-0b490f98cda4 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.701123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b26ce-bee8-4b0a-ab69-0b490f98cda4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:43:59.701428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b26ce-bee8-4b0a-ab69-0b490f98cda4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:59.702792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b26ce-bee8-4b0a-ab69-0b490f98cda4 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:59.715111 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 422/809] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:43:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:43:59.807384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:43:59.815358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7415677-1d03-4eba-aafc-2d5f8359c3ff None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:43:59.855170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b6132c-9898-475e-a8db-fb50bc798fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:43:59.855821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6132c-9898-475e-a8db-fb50bc798fa7 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:43:59.856074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6132c-9898-475e-a8db-fb50bc798fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:43:59.856347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b6132c-9898-475e-a8db-fb50bc798fa7 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:43:59.857134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6132c-9898-475e-a8db-fb50bc798fa7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:43:59.858582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b6132c-9898-475e-a8db-fb50bc798fa7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:43:59.884858 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 388/810] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:43:59 2022] POST /identity/v3/users => generated 285 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:44:01.330730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87458561-9aa8-400e-ae03-5d4d5da825cc None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:01.331276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87458561-9aa8-400e-ae03-5d4d5da825cc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:01.331276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87458561-9aa8-400e-ae03-5d4d5da825cc None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:01.332622 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 423/811] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:01.337942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd12262-707a-481f-97a9-5980504ba3a6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:01.338359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd12262-707a-481f-97a9-5980504ba3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:01.338476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd12262-707a-481f-97a9-5980504ba3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:01.355996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fdd12262-707a-481f-97a9-5980504ba3a6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:01.357206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fdd12262-707a-481f-97a9-5980504ba3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:01.420244 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 389/812] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:44:01.691120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87458561-9aa8-400e-ae03-5d4d5da825cc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:01.787480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87458561-9aa8-400e-ae03-5d4d5da825cc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:01.820697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:01.821315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:01.821589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:01.821856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:01.831791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:44:01.836554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:01.837666 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:44:01.838527 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 424/813] 10.222.0.39 () {56 vars in 1120 bytes} [Sun Oct 23 00:44:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:44:01.843534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd12262-707a-481f-97a9-5980504ba3a6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:01.923246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:01.941377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd12262-707a-481f-97a9-5980504ba3a6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:01.979428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4206a20-9ac6-4bc3-8f2d-79689f4ce5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:01.980040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206a20-9ac6-4bc3-8f2d-79689f4ce5c8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:01.980335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206a20-9ac6-4bc3-8f2d-79689f4ce5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:01.980612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4206a20-9ac6-4bc3-8f2d-79689f4ce5c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:01.981271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4206a20-9ac6-4bc3-8f2d-79689f4ce5c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:01.981571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4206a20-9ac6-4bc3-8f2d-79689f4ce5c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:01.982923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4206a20-9ac6-4bc3-8f2d-79689f4ce5c8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:01.995406 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 390/814] 10.222.0.39 () {56 vars in 1115 bytes} [Sun Oct 23 00:44:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:01.996654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.042124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:02.059083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba2d9-a2c2-42bf-ada3-c74f906a4368 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.059360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba2d9-a2c2-42bf-ada3-c74f906a4368 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.059629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2ba2d9-a2c2-42bf-ada3-c74f906a4368 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.061018 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 391/815] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:44:02.065819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7396d1-bd07-4872-9081-ea8d555199a7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.066097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7396d1-bd07-4872-9081-ea8d555199a7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.066360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7396d1-bd07-4872-9081-ea8d555199a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.097121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb202b-e0d8-4bf9-bf2b-fad079cace50 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.133558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.134209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.134485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.134756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.146204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:02.147676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:44:02.163745 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:02.164845 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:44:02.165736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:44:02.165736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:44:02.165736 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:44:02.165736 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 425/816] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:44:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:44:02.170988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.197182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ec7396d1-bd07-4872-9081-ea8d555199a7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:02.198275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ec7396d1-bd07-4872-9081-ea8d555199a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:02.264091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:02.265430 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 392/817] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:44:02.270161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a036a0e-da83-4c31-912b-bd63204bd723 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.303486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.304075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.304377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.304648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.305277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:02.305572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:02.306916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:02.318157 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 426/818] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:44:02.323805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7396d1-bd07-4872-9081-ea8d555199a7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.395955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:02.417935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7396d1-bd07-4872-9081-ea8d555199a7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.450193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.450750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.451024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.451292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.458517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:02.459915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:44:02.470954 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:02.472082 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:44:02.473266 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 393/819] 10.222.0.39 () {56 vars in 1093 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:44:02.477915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.532631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.575817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb296fe5-8113-4b77-b1e9-47ce1a68f0e0 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.608142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.608771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.609056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.609327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.610047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:02.610587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:02.611749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:02.628238 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 427/820] 10.222.0.39 () {56 vars in 1133 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:02.633969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f21f9-c59d-4d40-aef5-cd052e6998da None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.635499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.651070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:02.665783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.666402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.666676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.666939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.669823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:02.679307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:44:02.684741 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:02.685878 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:44:02.686929 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 394/821] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:44:02.691630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.736865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493786f3-e7d7-43d6-a584-448cc7791ea1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.770960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.771555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.771837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.772105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.787344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8f1496-8243-40d4-9837-83d15d0c7f33 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.823365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.823968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.824245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.824541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.825201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:02.825591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:02.826856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:02.835628 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 395/822] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:44:02.838244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:02.839733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:02.842915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.854636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:02.903168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:02.939354 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 428/823] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:44:02.944960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe2c339-6204-491f-bef8-41e66686becd None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:02.945752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:02.977851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:02.978473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:02.978752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:02.979020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] PATH_INFO: `/v3/users/affb0b3b59234da7a651ddeb15c99c5f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:02.992278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] RBAC: Authorizing `identity:get_user(user_id=affb0b3b59234da7a651ddeb15c99c5f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:02.993669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:02.995257 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 396/824] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/users/affb0b3b59234da7a651ddeb15c99c5f => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:44:03.000972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:03.039994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bfad46-82fa-45f0-8c05-df354affc404 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:03.053884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:03.074315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:03.074942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:03.075217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:03.075488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:03.085159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:03.086586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:03.095792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895d3a8a-9d8d-4f65-a8ac-f14bc0ea4ee0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:44:03.102756 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:03.103770 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:44:03.105029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:44:03.105029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:44:03.105029 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:44:03.105029 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 429/825] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:44:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:44:03.109401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:03.112152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:03.130612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:03.131234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:03.131507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:03.132121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] PATH_INFO: `/v3/projects/010044d6306e43c2b258e0ad0760875d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:03.133231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] RBAC: Authorizing `identity:get_project(project_id=010044d6306e43c2b258e0ad0760875d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:03.134655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:03.136278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 397/826] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:03 2022] GET /identity/v3/projects/010044d6306e43c2b258e0ad0760875d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:44:03.141988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:03.206602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce93888d-c462-4b18-b3a8-0b26a7c7ae20 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:03.237948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6039929-ff30-417c-afce-8bb9ccc77871 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:03.238597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:03.239747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:03.240034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:03.240310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:03.240953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:03.241248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:03.242656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:03.253684 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 430/827] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:44:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:44:03.259523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:03.274655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:03.275300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:03.275573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:03.275842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] PATH_INFO: `/v3/projects/010044d6306e43c2b258e0ad0760875d/users/affb0b3b59234da7a651ddeb15c99c5f/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:03.277733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] RBAC: Authorizing `identity:create_grant(project_id=010044d6306e43c2b258e0ad0760875d, user_id=affb0b3b59234da7a651ddeb15c99c5f, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:03.279125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:03.285707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:03.292186 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 398/828] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:44:03 2022] PUT /identity/v3/projects/010044d6306e43c2b258e0ad0760875d/users/affb0b3b59234da7a651ddeb15c99c5f/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:44:03.362974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ca348-0e3d-480b-9dd9-c26a1a50170b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:03.398648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:03.399290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:03.399567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:03.399840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:03.406957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:03.408448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:44:03.420894 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:03.421907 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:44:03.423397 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 431/829] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:44:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:44:03.428754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:03.478393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:03.532891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481b2de1-181c-4fef-8706-9d453e5d3172 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:03.565305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57d3b5-cfa1-4e0a-8a46-dcbfe83bd34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:03.565902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57d3b5-cfa1-4e0a-8a46-dcbfe83bd34b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:03.566179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57d3b5-cfa1-4e0a-8a46-dcbfe83bd34b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:03.566445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57d3b5-cfa1-4e0a-8a46-dcbfe83bd34b None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:03.567127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57d3b5-cfa1-4e0a-8a46-dcbfe83bd34b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:03.567543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57d3b5-cfa1-4e0a-8a46-dcbfe83bd34b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:03.568972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57d3b5-cfa1-4e0a-8a46-dcbfe83bd34b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:03.579992 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 399/830] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:44:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:03.586484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:03.670063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:03.685154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b17e16-20b9-4835-8379-33ff9a11961f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:03.717792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356809af-5ee8-46be-8541-6de2abfd8391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:03.718389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356809af-5ee8-46be-8541-6de2abfd8391 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:03.718666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356809af-5ee8-46be-8541-6de2abfd8391 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:03.718929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356809af-5ee8-46be-8541-6de2abfd8391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:03.719647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356809af-5ee8-46be-8541-6de2abfd8391 None admin] RBAC: Adding query filter params (role.id=eda5bb304e03428dbc184a9019dc3cda, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=f04fb9e1d1bd42ac815078b61f6a17f8) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:03.719946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356809af-5ee8-46be-8541-6de2abfd8391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:03.721387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356809af-5ee8-46be-8541-6de2abfd8391 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:03.727590 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 432/831] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:44:03 2022] GET /identity/v3/role_assignments?user.id=f04fb9e1d1bd42ac815078b61f6a17f8&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=eda5bb304e03428dbc184a9019dc3cda => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:05.410391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d50b10-e2f0-4bc5-9450-4b3561bb6291 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:05.410922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d50b10-e2f0-4bc5-9450-4b3561bb6291 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:05.410922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d50b10-e2f0-4bc5-9450-4b3561bb6291 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:05.412243 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 400/832] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:05.417527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588977-8c67-443b-83ad-8064f5b87604 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:05.417976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588977-8c67-443b-83ad-8064f5b87604 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:05.418077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35588977-8c67-443b-83ad-8064f5b87604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:05.435661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-35588977-8c67-443b-83ad-8064f5b87604 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:05.436873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-35588977-8c67-443b-83ad-8064f5b87604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:05.500709 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 433/833] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:44:05.765177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d50b10-e2f0-4bc5-9450-4b3561bb6291 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:05.863615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d50b10-e2f0-4bc5-9450-4b3561bb6291 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:05.865892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:05.896462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:05.897076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:05.897346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:05.897606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:05.907179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:44:05.912470 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:05.913545 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:44:05.917409 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 401/834] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:44:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:44:05.919511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588977-8c67-443b-83ad-8064f5b87604 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:05.995821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:06.014874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35588977-8c67-443b-83ad-8064f5b87604 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:06.048061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:06.048613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:06.048886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:06.049151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:06.049797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:06.050091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:06.051424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:06.059032 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 434/835] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:44:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:06.064905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:06.160548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe72e4f0-d235-4000-befb-6cf05e2ba6a4 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:06.193159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:06.193762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:06.194032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:06.194292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] PATH_INFO: `/v3/users/affb0b3b59234da7a651ddeb15c99c5f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:06.195366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] RBAC: Authorizing `identity:get_user(user_id=affb0b3b59234da7a651ddeb15c99c5f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:06.196784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:06.198299 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 402/836] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:06 2022] GET /identity/v3/users/affb0b3b59234da7a651ddeb15c99c5f => generated 285 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:06.203926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:06.298862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcbd167-417c-4cce-bc57-34c002f745aa None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:06.331267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:06.331856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:06.332133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:06.332432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:06.344179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:06.398346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:06.399766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:06.493606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 435/837] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:44:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:44:06.499404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:06.529566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:06.595048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e16eee-5829-4241-ba43-b7105327e6f0 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:06.626759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57be9faf-d70a-49db-8a26-3b6f8870cff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:06.627362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be9faf-d70a-49db-8a26-3b6f8870cff0 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:06.627633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be9faf-d70a-49db-8a26-3b6f8870cff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:06.627896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be9faf-d70a-49db-8a26-3b6f8870cff0 None admin] PATH_INFO: `/v3/projects/010044d6306e43c2b258e0ad0760875d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:06.628984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be9faf-d70a-49db-8a26-3b6f8870cff0 None admin] RBAC: Authorizing `identity:get_project(project_id=010044d6306e43c2b258e0ad0760875d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:06.630976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57be9faf-d70a-49db-8a26-3b6f8870cff0 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:06.632823 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 403/838] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:06 2022] GET /identity/v3/projects/010044d6306e43c2b258e0ad0760875d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:06.638858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:06.679624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:06.732583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369cb18-8b12-4c7f-9050-bba3d6a36842 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:06.765485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28d03d-d3c8-4613-983b-b02e51cf174c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:06.766066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28d03d-d3c8-4613-983b-b02e51cf174c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:06.766345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28d03d-d3c8-4613-983b-b02e51cf174c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:06.766610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28d03d-d3c8-4613-983b-b02e51cf174c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:06.767319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28d03d-d3c8-4613-983b-b02e51cf174c None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=010044d6306e43c2b258e0ad0760875d, user.id=affb0b3b59234da7a651ddeb15c99c5f) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:06.767620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28d03d-d3c8-4613-983b-b02e51cf174c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:06.769047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28d03d-d3c8-4613-983b-b02e51cf174c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:06.774526 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 436/839] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:44:06 2022] GET /identity/v3/role_assignments?user.id=affb0b3b59234da7a651ddeb15c99c5f&scope.project.id=010044d6306e43c2b258e0ad0760875d&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:08.892500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2b858-64dc-4c21-97cf-23304612b9c2 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:08.893009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2b858-64dc-4c21-97cf-23304612b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:08.893101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd2b858-64dc-4c21-97cf-23304612b9c2 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:08.894359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 404/840] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:08.900161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d032a7-78cf-4399-a4c7-5c5580db58c0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:08.900593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d032a7-78cf-4399-a4c7-5c5580db58c0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:08.900739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d032a7-78cf-4399-a4c7-5c5580db58c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:08.918697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-73d032a7-78cf-4399-a4c7-5c5580db58c0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:08.919844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-73d032a7-78cf-4399-a4c7-5c5580db58c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:08.983212 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 437/841] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:09.250442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2b858-64dc-4c21-97cf-23304612b9c2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:09.343007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:09.347009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd2b858-64dc-4c21-97cf-23304612b9c2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:09.379713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a012736-38b9-424f-b7cc-00963b2f2798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:09.380340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a012736-38b9-424f-b7cc-00963b2f2798 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:09.380617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a012736-38b9-424f-b7cc-00963b2f2798 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:09.380884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a012736-38b9-424f-b7cc-00963b2f2798 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:09.382216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a012736-38b9-424f-b7cc-00963b2f2798 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:09.383600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a012736-38b9-424f-b7cc-00963b2f2798 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:09.385114 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 405/842] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:44:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:09.390655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d032a7-78cf-4399-a4c7-5c5580db58c0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:09.423305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:09.486323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d032a7-78cf-4399-a4c7-5c5580db58c0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:09.518544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ef794-665c-48a8-8965-eccbed73bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:09.519081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ef794-665c-48a8-8965-eccbed73bb13 None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:09.519359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ef794-665c-48a8-8965-eccbed73bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:09.519632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ef794-665c-48a8-8965-eccbed73bb13 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:09.520827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ef794-665c-48a8-8965-eccbed73bb13 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:09.522243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ef794-665c-48a8-8965-eccbed73bb13 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:09.543017 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 438/843] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:44:09 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:44:11.674952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f3a0-18d8-47fb-b452-129fa213517d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:11.675518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f3a0-18d8-47fb-b452-129fa213517d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:11.675518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db52f3a0-18d8-47fb-b452-129fa213517d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:11.677449 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 406/844] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:11.682671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013866b-5fa3-4bb9-ab3a-3b7c9970ed33 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:11.683101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013866b-5fa3-4bb9-ab3a-3b7c9970ed33 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:11.683227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013866b-5fa3-4bb9-ab3a-3b7c9970ed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:11.701454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3013866b-5fa3-4bb9-ab3a-3b7c9970ed33 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:11.702623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3013866b-5fa3-4bb9-ab3a-3b7c9970ed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:11.767608 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 439/845] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:12.034214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f3a0-18d8-47fb-b452-129fa213517d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:12.130878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db52f3a0-18d8-47fb-b452-129fa213517d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:12.162790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:12.163387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:12.163658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:12.163921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] PATH_INFO: `/v3/roles/633fd4ad499a4f62a5a2cbac0eb861ce` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:12.165280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] RBAC: Authorizing `identity:get_role(role_id=633fd4ad499a4f62a5a2cbac0eb861ce)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:12.166621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:12.167878 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 407/846] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:12 2022] GET /identity/v3/roles/633fd4ad499a4f62a5a2cbac0eb861ce => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:12.173500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013866b-5fa3-4bb9-ab3a-3b7c9970ed33 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:12.196463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:12.270033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3013866b-5fa3-4bb9-ab3a-3b7c9970ed33 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:12.302508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:12.303132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:12.303410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:12.303680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] PATH_INFO: `/v3/users/c922703bd98645edbd3daa08a9d7f1f8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:12.316400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] RBAC: Authorizing `identity:get_user(user_id=c922703bd98645edbd3daa08a9d7f1f8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:12.318340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:12.319903 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 440/847] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:12 2022] GET /identity/v3/users/c922703bd98645edbd3daa08a9d7f1f8 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:12.325588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:12.358900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:12.421411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c61a1-44b5-409e-812a-82deb530a233 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:12.454563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed37295-9b5b-45fb-a03a-dff572771ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:12.455175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed37295-9b5b-45fb-a03a-dff572771ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:12.455454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed37295-9b5b-45fb-a03a-dff572771ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:12.455728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed37295-9b5b-45fb-a03a-dff572771ba9 None admin] PATH_INFO: `/v3/projects/010044d6306e43c2b258e0ad0760875d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:12.456850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed37295-9b5b-45fb-a03a-dff572771ba9 None admin] RBAC: Authorizing `identity:get_project(project_id=010044d6306e43c2b258e0ad0760875d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:12.458251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed37295-9b5b-45fb-a03a-dff572771ba9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:12.459833 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 408/848] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:12 2022] GET /identity/v3/projects/010044d6306e43c2b258e0ad0760875d => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:12.465511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:12.544381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:12.563374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7b19c-ea02-4273-a3a6-2bba04b65e20 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:12.596484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0f7b6-7150-46e5-8154-5f1be5bf0a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:12.597127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0f7b6-7150-46e5-8154-5f1be5bf0a68 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:12.597405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0f7b6-7150-46e5-8154-5f1be5bf0a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:12.597675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0f7b6-7150-46e5-8154-5f1be5bf0a68 None admin] PATH_INFO: `/v3/projects/010044d6306e43c2b258e0ad0760875d/users/c922703bd98645edbd3daa08a9d7f1f8/roles/633fd4ad499a4f62a5a2cbac0eb861ce` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:12.600367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0f7b6-7150-46e5-8154-5f1be5bf0a68 None admin] RBAC: Authorizing `identity:create_grant(project_id=010044d6306e43c2b258e0ad0760875d, user_id=c922703bd98645edbd3daa08a9d7f1f8, role_id=633fd4ad499a4f62a5a2cbac0eb861ce)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:12.601786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0f7b6-7150-46e5-8154-5f1be5bf0a68 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:12.614557 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 441/849] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:44:12 2022] PUT /identity/v3/projects/010044d6306e43c2b258e0ad0760875d/users/c922703bd98645edbd3daa08a9d7f1f8/roles/633fd4ad499a4f62a5a2cbac0eb861ce => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:44:14.731153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfbe01-261b-4996-87e8-4ca9befd0f03 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:14.731684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfbe01-261b-4996-87e8-4ca9befd0f03 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:14.731684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcfbe01-261b-4996-87e8-4ca9befd0f03 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:14.733078 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 409/850] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:14.738291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9b15-3e5b-42a4-a274-2d554413dd50 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:14.738721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9b15-3e5b-42a4-a274-2d554413dd50 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:14.738870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3be9b15-3e5b-42a4-a274-2d554413dd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:14.756230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b3be9b15-3e5b-42a4-a274-2d554413dd50 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:14.757668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b3be9b15-3e5b-42a4-a274-2d554413dd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:14.819867 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 442/851] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:15.086258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfbe01-261b-4996-87e8-4ca9befd0f03 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:15.182025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcfbe01-261b-4996-87e8-4ca9befd0f03 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:15.184221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:15.215588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:15.216189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:15.216494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:15.216759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] PATH_INFO: `/v3/roles/633fd4ad499a4f62a5a2cbac0eb861ce` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:15.218106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] RBAC: Authorizing `identity:get_role(role_id=633fd4ad499a4f62a5a2cbac0eb861ce)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:15.219427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:15.220730 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 410/852] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:15 2022] GET /identity/v3/roles/633fd4ad499a4f62a5a2cbac0eb861ce => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:15.226342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be9b15-3e5b-42a4-a274-2d554413dd50 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:15.291257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:15.322639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3be9b15-3e5b-42a4-a274-2d554413dd50 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:15.355011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:15.355561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:15.355837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:15.356106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] PATH_INFO: `/v3/users/c922703bd98645edbd3daa08a9d7f1f8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:15.357217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] RBAC: Authorizing `identity:get_user(user_id=c922703bd98645edbd3daa08a9d7f1f8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:15.358629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:15.360169 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 443/853] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:15 2022] GET /identity/v3/users/c922703bd98645edbd3daa08a9d7f1f8 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:15.365866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:15.460036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc8e554-4d7a-4c15-9281-42cf129f57bb None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:15.492578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:15.493169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:15.493445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:15.493741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:15.532056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:15.557666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:15.559065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:15.651161 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 411/854] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:44:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:44:15.656863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:15.752054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e414d3e4-8716-40ad-ad63-8e00ec3af6fd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:15.784869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0d467-9581-458e-b658-5fa8921949f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:15.785489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0d467-9581-458e-b658-5fa8921949f1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:15.785770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0d467-9581-458e-b658-5fa8921949f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:15.786042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0d467-9581-458e-b658-5fa8921949f1 None admin] PATH_INFO: `/v3/projects/010044d6306e43c2b258e0ad0760875d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:15.787146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0d467-9581-458e-b658-5fa8921949f1 None admin] RBAC: Authorizing `identity:get_project(project_id=010044d6306e43c2b258e0ad0760875d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:15.788589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0d467-9581-458e-b658-5fa8921949f1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:15.790199 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 444/855] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:15 2022] GET /identity/v3/projects/010044d6306e43c2b258e0ad0760875d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:15.796398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:15.864471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:15.892850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b0475-70ba-46f1-8424-aa4f7fb8e7c8 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:15.924675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343a82d-80e5-4f43-9d6e-504035b21f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:15.925207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a82d-80e5-4f43-9d6e-504035b21f3a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:15.925482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a82d-80e5-4f43-9d6e-504035b21f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:15.925753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343a82d-80e5-4f43-9d6e-504035b21f3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:15.926463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a82d-80e5-4f43-9d6e-504035b21f3a None admin] RBAC: Adding query filter params (role.id=633fd4ad499a4f62a5a2cbac0eb861ce, scope.project.id=010044d6306e43c2b258e0ad0760875d, user.id=c922703bd98645edbd3daa08a9d7f1f8) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:15.926765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a82d-80e5-4f43-9d6e-504035b21f3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:15.928155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343a82d-80e5-4f43-9d6e-504035b21f3a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:15.933622 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 412/856] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:44:15 2022] GET /identity/v3/role_assignments?user.id=c922703bd98645edbd3daa08a9d7f1f8&scope.project.id=010044d6306e43c2b258e0ad0760875d&role.id=633fd4ad499a4f62a5a2cbac0eb861ce => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:18.042170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906b3a9-68bd-4951-a0cb-bc3e3c80e19a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:18.042689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906b3a9-68bd-4951-a0cb-bc3e3c80e19a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:18.042689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906b3a9-68bd-4951-a0cb-bc3e3c80e19a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:18.044027 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 445/857] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:18.049276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac06bf-d2d7-457a-892e-7832ec62df5e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:18.049665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac06bf-d2d7-457a-892e-7832ec62df5e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:18.049812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfac06bf-d2d7-457a-892e-7832ec62df5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:18.067756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bfac06bf-d2d7-457a-892e-7832ec62df5e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:18.068935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bfac06bf-d2d7-457a-892e-7832ec62df5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:18.131733 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 413/858] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:18.396183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906b3a9-68bd-4951-a0cb-bc3e3c80e19a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:18.405452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:18.493395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906b3a9-68bd-4951-a0cb-bc3e3c80e19a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:18.525755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd42869-0d6a-4773-90c0-62651c237a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:18.526360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42869-0d6a-4773-90c0-62651c237a9b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:18.526633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42869-0d6a-4773-90c0-62651c237a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:18.526899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd42869-0d6a-4773-90c0-62651c237a9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:18.527976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd42869-0d6a-4773-90c0-62651c237a9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:18.529412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd42869-0d6a-4773-90c0-62651c237a9b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:18.530937 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 446/859] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:44:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:18.536945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac06bf-d2d7-457a-892e-7832ec62df5e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:18.598150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:18.632714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfac06bf-d2d7-457a-892e-7832ec62df5e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:18.664381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b26941-beb8-4e7e-a7e4-42960e2416a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:18.664910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26941-beb8-4e7e-a7e4-42960e2416a7 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:18.665188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26941-beb8-4e7e-a7e4-42960e2416a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:18.665457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b26941-beb8-4e7e-a7e4-42960e2416a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:18.666097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b26941-beb8-4e7e-a7e4-42960e2416a7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:18.667506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b26941-beb8-4e7e-a7e4-42960e2416a7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:18.693264 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 414/860] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:44:18 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:44:20.845299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a488c6-0c90-4383-b7fd-1afdf5102504 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:20.845737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a488c6-0c90-4383-b7fd-1afdf5102504 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:20.845851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a488c6-0c90-4383-b7fd-1afdf5102504 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:20.847156 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 447/861] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:20.853101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28e31-d28c-4909-953f-99f5f17489c9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:20.853499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28e31-d28c-4909-953f-99f5f17489c9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:20.853653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e28e31-d28c-4909-953f-99f5f17489c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:20.872419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d8e28e31-d28c-4909-953f-99f5f17489c9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:20.873569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d8e28e31-d28c-4909-953f-99f5f17489c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:20.940606 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 415/862] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:21.211379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a488c6-0c90-4383-b7fd-1afdf5102504 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:21.284040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:21.310167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a488c6-0c90-4383-b7fd-1afdf5102504 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:21.342061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af3c6ff-b3be-4960-a8f2-4135f69995ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:21.342665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3c6ff-b3be-4960-a8f2-4135f69995ef None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:21.342944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3c6ff-b3be-4960-a8f2-4135f69995ef None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:21.343212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af3c6ff-b3be-4960-a8f2-4135f69995ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:21.344329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3c6ff-b3be-4960-a8f2-4135f69995ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:21.345732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af3c6ff-b3be-4960-a8f2-4135f69995ef None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:21.347251 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 448/863] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:44:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:21.353330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e28e31-d28c-4909-953f-99f5f17489c9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:21.399160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:21.448099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e28e31-d28c-4909-953f-99f5f17489c9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:21.480842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee01a28-c9e9-492a-9733-4b2226acfdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:21.481377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee01a28-c9e9-492a-9733-4b2226acfdc4 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:21.481649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee01a28-c9e9-492a-9733-4b2226acfdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:21.481915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee01a28-c9e9-492a-9733-4b2226acfdc4 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:21.482552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee01a28-c9e9-492a-9733-4b2226acfdc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:21.483934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee01a28-c9e9-492a-9733-4b2226acfdc4 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:21.503921 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 416/864] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:44:21 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:44:23.646085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45865190-c9e6-45a4-a537-0ce58b5ce2bc None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:23.646612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45865190-c9e6-45a4-a537-0ce58b5ce2bc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:23.646731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45865190-c9e6-45a4-a537-0ce58b5ce2bc None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:23.647968 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 449/865] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:23.653190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930d935-aafe-4fe2-b961-0ea172bfed1e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:23.653571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930d935-aafe-4fe2-b961-0ea172bfed1e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:23.653735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930d935-aafe-4fe2-b961-0ea172bfed1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:23.671481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4930d935-aafe-4fe2-b961-0ea172bfed1e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:23.672685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4930d935-aafe-4fe2-b961-0ea172bfed1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:23.736059 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 417/866] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:24.003949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45865190-c9e6-45a4-a537-0ce58b5ce2bc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:24.101010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45865190-c9e6-45a4-a537-0ce58b5ce2bc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:24.133241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:24.133853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:24.134131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:24.134403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:24.143946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:44:24.148713 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:24.149780 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:44:24.150616 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 450/867] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:44:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:44:24.155788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930d935-aafe-4fe2-b961-0ea172bfed1e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:24.188289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:24.251428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930d935-aafe-4fe2-b961-0ea172bfed1e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:24.283191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:24.283729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:24.284005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:24.284272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:24.284957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:24.285257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:24.287129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:24.294742 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 418/868] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:44:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:24.300463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:24.304423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:24.397375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef3f01-22cc-428c-886e-956fdf7ca97b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:24.431281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:24.431896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:24.432174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:24.432477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] PATH_INFO: `/v3/users/07ea578f39aa4fba94521290826bac50` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:24.445210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] RBAC: Authorizing `identity:get_user(user_id=07ea578f39aa4fba94521290826bac50)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:24.446632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:24.448202 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 451/869] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:24 2022] GET /identity/v3/users/07ea578f39aa4fba94521290826bac50 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:24.453930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:24.546448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:24.552553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2dbd0f-b04b-48b7-b719-26f59e28a459 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:24.584384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9433fca-a1bb-4a0a-b106-c8e42ab24784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:24.585001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433fca-a1bb-4a0a-b106-c8e42ab24784 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:24.585352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433fca-a1bb-4a0a-b106-c8e42ab24784 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:24.585551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9433fca-a1bb-4a0a-b106-c8e42ab24784 None admin] PATH_INFO: `/v3/projects/cd63e52a036a4188b8abee816208c0e5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:24.586642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9433fca-a1bb-4a0a-b106-c8e42ab24784 None admin] RBAC: Authorizing `identity:get_project(project_id=cd63e52a036a4188b8abee816208c0e5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:24.588041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9433fca-a1bb-4a0a-b106-c8e42ab24784 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:24.589654 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 419/870] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:24 2022] GET /identity/v3/projects/cd63e52a036a4188b8abee816208c0e5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:24.595534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:24.652400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:24.691812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d6e39-a9cd-4e3d-813e-fc13410bb368 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:24.724675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181029de-0397-4889-808b-ed67f14ca0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:24.725316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181029de-0397-4889-808b-ed67f14ca0f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:24.725593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181029de-0397-4889-808b-ed67f14ca0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:24.725864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181029de-0397-4889-808b-ed67f14ca0f5 None admin] PATH_INFO: `/v3/projects/cd63e52a036a4188b8abee816208c0e5/users/07ea578f39aa4fba94521290826bac50/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:24.727720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181029de-0397-4889-808b-ed67f14ca0f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=cd63e52a036a4188b8abee816208c0e5, user_id=07ea578f39aa4fba94521290826bac50, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:24.729168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181029de-0397-4889-808b-ed67f14ca0f5 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:24.741452 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 452/871] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:44:24 2022] PUT /identity/v3/projects/cd63e52a036a4188b8abee816208c0e5/users/07ea578f39aa4fba94521290826bac50/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:44:26.833566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fbeec-aae0-4fa6-b722-46ddd61ae07a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:26.833968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fbeec-aae0-4fa6-b722-46ddd61ae07a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:26.834113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fbeec-aae0-4fa6-b722-46ddd61ae07a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:26.835467 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 420/872] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:26.840662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dae32-c0af-458b-8786-2b11e2596643 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:26.841064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dae32-c0af-458b-8786-2b11e2596643 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:26.841208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8dae32-c0af-458b-8786-2b11e2596643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:26.859383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6c8dae32-c0af-458b-8786-2b11e2596643 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:26.860595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6c8dae32-c0af-458b-8786-2b11e2596643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:26.924606 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 453/873] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:27.194032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fbeec-aae0-4fa6-b722-46ddd61ae07a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:27.291474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fbeec-aae0-4fa6-b722-46ddd61ae07a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:27.324438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:27.325052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:27.325328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:27.325595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:27.335892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:44:27.340617 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:27.344401 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:44:27.345359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 421/874] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:44:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:44:27.347553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dae32-c0af-458b-8786-2b11e2596643 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:27.396998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:27.445892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8dae32-c0af-458b-8786-2b11e2596643 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:27.480223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:27.480845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:27.481126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:27.481398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:27.482061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:27.482363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:27.483715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:27.491666 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 454/875] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:44:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:27.497733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:27.523527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:27.594538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9eda7-fcbb-4a9e-80a0-d2e920174a27 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:27.626753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:27.627369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:27.627646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:27.627916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] PATH_INFO: `/v3/users/07ea578f39aa4fba94521290826bac50` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:27.629031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] RBAC: Authorizing `identity:get_user(user_id=07ea578f39aa4fba94521290826bac50)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:27.630952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:27.632793 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 422/876] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:27 2022] GET /identity/v3/users/07ea578f39aa4fba94521290826bac50 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:27.638475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:27.734194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9615d-cb67-4eca-aad6-66cd23708059 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:27.746283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:27.766444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:27.767041 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:27.767321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:27.767593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:27.832416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:27.834453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:27.929064 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 455/877] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:44:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:44:27.934918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:28.023512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:28.032495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27035b44-0102-4ffc-a376-2d4592d80880 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:28.064367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087a25c7-1d2e-4eed-bac5-be762828688d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:28.065000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a25c7-1d2e-4eed-bac5-be762828688d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:28.065284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a25c7-1d2e-4eed-bac5-be762828688d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:28.065537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087a25c7-1d2e-4eed-bac5-be762828688d None admin] PATH_INFO: `/v3/projects/cd63e52a036a4188b8abee816208c0e5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:28.066612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a25c7-1d2e-4eed-bac5-be762828688d None admin] RBAC: Authorizing `identity:get_project(project_id=cd63e52a036a4188b8abee816208c0e5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:28.068557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087a25c7-1d2e-4eed-bac5-be762828688d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:28.070131 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 423/878] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:27 2022] GET /identity/v3/projects/cd63e52a036a4188b8abee816208c0e5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:28.076094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:28.090068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:28.171010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27839a26-a6b6-4a7e-9cb1-6ccfdea7be97 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:28.203586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8370d-e81d-4c64-8296-0f2c1f063d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:28.204174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8370d-e81d-4c64-8296-0f2c1f063d7e None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:28.204482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8370d-e81d-4c64-8296-0f2c1f063d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:28.204755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8370d-e81d-4c64-8296-0f2c1f063d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:28.205465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8370d-e81d-4c64-8296-0f2c1f063d7e None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=cd63e52a036a4188b8abee816208c0e5, user.id=07ea578f39aa4fba94521290826bac50) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:28.205767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8370d-e81d-4c64-8296-0f2c1f063d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:28.207163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8370d-e81d-4c64-8296-0f2c1f063d7e None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:28.212678 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 456/879] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:44:28 2022] GET /identity/v3/role_assignments?user.id=07ea578f39aa4fba94521290826bac50&scope.project.id=cd63e52a036a4188b8abee816208c0e5&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:30.343367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56727dda-1eb2-420c-8672-d84a637289bd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:30.343939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56727dda-1eb2-420c-8672-d84a637289bd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:30.343939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56727dda-1eb2-420c-8672-d84a637289bd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:30.345268 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 424/880] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:30.351023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cf6f2-389c-446a-b1c6-1ecad4e52ba4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:30.351420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cf6f2-389c-446a-b1c6-1ecad4e52ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:30.351558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13cf6f2-389c-446a-b1c6-1ecad4e52ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:30.369946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c13cf6f2-389c-446a-b1c6-1ecad4e52ba4 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:30.371123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c13cf6f2-389c-446a-b1c6-1ecad4e52ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:30.434714 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 457/881] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:30.700701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56727dda-1eb2-420c-8672-d84a637289bd None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:30.796275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56727dda-1eb2-420c-8672-d84a637289bd None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:30.832474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73791-5c45-4db2-bc48-41e7300c01d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:30.833722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73791-5c45-4db2-bc48-41e7300c01d9 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:30.834000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73791-5c45-4db2-bc48-41e7300c01d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:30.834270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73791-5c45-4db2-bc48-41e7300c01d9 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:30.834925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d73791-5c45-4db2-bc48-41e7300c01d9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:30.836267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d73791-5c45-4db2-bc48-41e7300c01d9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:30.851486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 425/882] 10.222.0.39 () {60 vars in 1138 bytes} [Sun Oct 23 00:44:30 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:44:32.975108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed147d-28d4-49fe-8136-19912f814fe1 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:32.975655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed147d-28d4-49fe-8136-19912f814fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:32.975655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed147d-28d4-49fe-8136-19912f814fe1 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:32.977039 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 458/883] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:32.982116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff624a1-4394-4473-b182-077edc45adff None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:32.982547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff624a1-4394-4473-b182-077edc45adff None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:32.982655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff624a1-4394-4473-b182-077edc45adff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:33.000859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-dff624a1-4394-4473-b182-077edc45adff None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:33.002029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dff624a1-4394-4473-b182-077edc45adff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:33.016547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:33.067836 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 426/884] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:33.332025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed147d-28d4-49fe-8136-19912f814fe1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:33.340849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:33.430484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed147d-28d4-49fe-8136-19912f814fe1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:33.462487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d508b8-0d81-4fb1-aa45-d14c41f3d87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:33.463097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d508b8-0d81-4fb1-aa45-d14c41f3d87c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:33.463374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d508b8-0d81-4fb1-aa45-d14c41f3d87c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:33.463646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d508b8-0d81-4fb1-aa45-d14c41f3d87c None admin] PATH_INFO: `/v3/domains/09e3900a4cb04205a98ce727de462439` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:33.474209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d508b8-0d81-4fb1-aa45-d14c41f3d87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=09e3900a4cb04205a98ce727de462439)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:33.475626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d508b8-0d81-4fb1-aa45-d14c41f3d87c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:33.477252 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 459/885] 10.222.0.39 () {56 vars in 1183 bytes} [Sun Oct 23 00:44:33 2022] GET /identity/v3/domains/09e3900a4cb04205a98ce727de462439 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:33.483145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff624a1-4394-4473-b182-077edc45adff None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:33.576976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff624a1-4394-4473-b182-077edc45adff None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:33.610001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7df045-7db1-4a34-b7be-4e9b81721253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:33.610539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7df045-7db1-4a34-b7be-4e9b81721253 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:33.610815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7df045-7db1-4a34-b7be-4e9b81721253 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:33.611082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7df045-7db1-4a34-b7be-4e9b81721253 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:33.611725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7df045-7db1-4a34-b7be-4e9b81721253 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:33.613154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7df045-7db1-4a34-b7be-4e9b81721253 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:33.638190 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 427/886] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:44:33 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:44:35.796644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1051a-d371-42ca-b1ea-a7081b889970 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:35.797171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1051a-d371-42ca-b1ea-a7081b889970 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:35.797171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e1051a-d371-42ca-b1ea-a7081b889970 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:35.798500 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 460/887] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:35.803677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6e2b0-2ff9-4fd9-bc85-c6a06428a3a9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:35.804173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6e2b0-2ff9-4fd9-bc85-c6a06428a3a9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:35.804277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6e2b0-2ff9-4fd9-bc85-c6a06428a3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:35.810384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:35.824048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6fb6e2b0-2ff9-4fd9-bc85-c6a06428a3a9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:35.825283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6fb6e2b0-2ff9-4fd9-bc85-c6a06428a3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:35.888129 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 428/888] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:44:36.154883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1051a-d371-42ca-b1ea-a7081b889970 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:36.218299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:36.252199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e1051a-d371-42ca-b1ea-a7081b889970 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:36.285620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c9588-b102-4904-8ce5-57a7777ea8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:36.286261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c9588-b102-4904-8ce5-57a7777ea8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:36.286535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c9588-b102-4904-8ce5-57a7777ea8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:36.286803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c9588-b102-4904-8ce5-57a7777ea8e7 None admin] PATH_INFO: `/v3/domains/09e3900a4cb04205a98ce727de462439` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:36.287895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c9588-b102-4904-8ce5-57a7777ea8e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=09e3900a4cb04205a98ce727de462439)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:36.289329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c9588-b102-4904-8ce5-57a7777ea8e7 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:36.290868 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 461/889] 10.222.0.39 () {56 vars in 1183 bytes} [Sun Oct 23 00:44:36 2022] GET /identity/v3/domains/09e3900a4cb04205a98ce727de462439 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:36.296528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6e2b0-2ff9-4fd9-bc85-c6a06428a3a9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:36.389792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6e2b0-2ff9-4fd9-bc85-c6a06428a3a9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:36.417577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:36.422847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8b1d5-aecf-41fb-9f6a-e71a3d328188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:36.423446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8b1d5-aecf-41fb-9f6a-e71a3d328188 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:36.423718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8b1d5-aecf-41fb-9f6a-e71a3d328188 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:36.423983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8b1d5-aecf-41fb-9f6a-e71a3d328188 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:36.424660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8b1d5-aecf-41fb-9f6a-e71a3d328188 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:36.426056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8b1d5-aecf-41fb-9f6a-e71a3d328188 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:36.446394 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 429/890] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:44:36 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:44:38.588518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597778e1-dc17-470c-bdeb-0b4efed0c66f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:38.589063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597778e1-dc17-470c-bdeb-0b4efed0c66f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:38.589063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597778e1-dc17-470c-bdeb-0b4efed0c66f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:38.590370 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 462/891] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:38.595435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8dc7-446e-4c48-8a32-c9c72ebba7f9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:38.595809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8dc7-446e-4c48-8a32-c9c72ebba7f9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:38.595981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c8dc7-446e-4c48-8a32-c9c72ebba7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:38.614695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5b5c8dc7-446e-4c48-8a32-c9c72ebba7f9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:38.615849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b5c8dc7-446e-4c48-8a32-c9c72ebba7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:38.678599 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 430/892] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:38.946741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597778e1-dc17-470c-bdeb-0b4efed0c66f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:39.043185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597778e1-dc17-470c-bdeb-0b4efed0c66f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:39.075926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:39.076562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:39.076835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:39.077102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:39.078857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:39.088045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:44:39.092694 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:39.093849 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:44:39.094959 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 463/893] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:44:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:44:39.099904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c8dc7-446e-4c48-8a32-c9c72ebba7f9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:39.194296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c8dc7-446e-4c48-8a32-c9c72ebba7f9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:39.211012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:39.228478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:39.229072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:39.229346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:39.229613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:39.230288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:39.230584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:39.231927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:39.239686 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 431/894] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:44:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:39.245511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:39.339826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bbf4c-84de-4331-a7e1-7a926e43556a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:39.372342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:39.372958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:39.373232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:39.373500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] PATH_INFO: `/v3/users/bf72f22f0f574e04bc76fc9089e97ac4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:39.387332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] RBAC: Authorizing `identity:get_user(user_id=bf72f22f0f574e04bc76fc9089e97ac4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:39.388778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:39.390358 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 464/895] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:39 2022] GET /identity/v3/users/bf72f22f0f574e04bc76fc9089e97ac4 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:39.396833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:39.492730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00463873-7f1e-4a91-94f9-f7a274811ec7 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:39.525968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7666b854-4a3b-41ef-9957-9b3e89f6131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:39.526589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666b854-4a3b-41ef-9957-9b3e89f6131a None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:39.526867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666b854-4a3b-41ef-9957-9b3e89f6131a None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:39.527134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7666b854-4a3b-41ef-9957-9b3e89f6131a None admin] PATH_INFO: `/v3/projects/0da6bbbf30744afab58fab5feeb25970` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:39.528235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666b854-4a3b-41ef-9957-9b3e89f6131a None admin] RBAC: Authorizing `identity:get_project(project_id=0da6bbbf30744afab58fab5feeb25970)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:39.529669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7666b854-4a3b-41ef-9957-9b3e89f6131a None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:39.531279 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 432/896] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:39 2022] GET /identity/v3/projects/0da6bbbf30744afab58fab5feeb25970 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:39.537159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:39.587973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:39.634016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f1c1c-b994-432f-8ba0-e97a8b4f7e7d None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:39.666212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac76e10e-352b-4be0-a141-fc20ba7294f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:39.666863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76e10e-352b-4be0-a141-fc20ba7294f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:39.667140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76e10e-352b-4be0-a141-fc20ba7294f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:39.667410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76e10e-352b-4be0-a141-fc20ba7294f8 None admin] PATH_INFO: `/v3/projects/0da6bbbf30744afab58fab5feeb25970/users/bf72f22f0f574e04bc76fc9089e97ac4/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:39.669291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76e10e-352b-4be0-a141-fc20ba7294f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0da6bbbf30744afab58fab5feeb25970, user_id=bf72f22f0f574e04bc76fc9089e97ac4, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:39.670705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac76e10e-352b-4be0-a141-fc20ba7294f8 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:39.684029 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 465/897] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:44:39 2022] PUT /identity/v3/projects/0da6bbbf30744afab58fab5feeb25970/users/bf72f22f0f574e04bc76fc9089e97ac4/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:44:41.795283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fbd38-adc7-4ad6-abc2-9024a123aa73 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:41.796955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fbd38-adc7-4ad6-abc2-9024a123aa73 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:41.796955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fbd38-adc7-4ad6-abc2-9024a123aa73 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:41.797330 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 433/898] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:44:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:44:41.802506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddef3c-612e-4406-884a-206ae1a61ed5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:41.802903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddef3c-612e-4406-884a-206ae1a61ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:41.803046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddef3c-612e-4406-884a-206ae1a61ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:41.820693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-09ddef3c-612e-4406-884a-206ae1a61ed5 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:44:41.821852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-09ddef3c-612e-4406-884a-206ae1a61ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:44:41.885910 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 466/899] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:44:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:44:42.155932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fbd38-adc7-4ad6-abc2-9024a123aa73 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:42.204661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:42.254903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fbd38-adc7-4ad6-abc2-9024a123aa73 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:42.287935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:42.288573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:42.288854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:42.289125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:42.298684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:44:42.303457 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:44:42.304621 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:44:42.305726 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 434/900] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:44:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:44:42.310626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddef3c-612e-4406-884a-206ae1a61ed5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:42.351890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:42.406895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddef3c-612e-4406-884a-206ae1a61ed5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:42.439259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:42.439853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:42.440135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:42.440435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:42.441091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:42.441393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:42.443336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:42.450973 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 467/901] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:44:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:42.456798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:42.550709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2d338-999c-421c-9004-62a3d3258b87 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:42.559901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:42.584315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:42.584929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:42.585201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:42.585466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] PATH_INFO: `/v3/users/bf72f22f0f574e04bc76fc9089e97ac4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:42.586542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] RBAC: Authorizing `identity:get_user(user_id=bf72f22f0f574e04bc76fc9089e97ac4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:42.587933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:42.589494 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 435/902] 10.222.0.39 () {56 vars in 1177 bytes} [Sun Oct 23 00:44:42 2022] GET /identity/v3/users/bf72f22f0f574e04bc76fc9089e97ac4 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:42.595101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:42.691445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858caa2-6386-42ff-8753-baf008d72787 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:42.694203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:42.724394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:42.724985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:42.725260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:42.725528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:42.789302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:42.790715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:42.883314 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 468/903] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:44:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:44:42.889374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:42.982793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a932155-4bc2-4033-9bf2-2f8fa875b35b None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:43.014824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f71c6-9abb-4725-ac10-c1ad075009cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:43.015424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f71c6-9abb-4725-ac10-c1ad075009cf None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:43.015697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f71c6-9abb-4725-ac10-c1ad075009cf None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:43.015962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f71c6-9abb-4725-ac10-c1ad075009cf None admin] PATH_INFO: `/v3/projects/0da6bbbf30744afab58fab5feeb25970` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:43.017060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f71c6-9abb-4725-ac10-c1ad075009cf None admin] RBAC: Authorizing `identity:get_project(project_id=0da6bbbf30744afab58fab5feeb25970)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:43.018454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f71c6-9abb-4725-ac10-c1ad075009cf None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:43.020619 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 436/904] 10.222.0.39 () {56 vars in 1186 bytes} [Sun Oct 23 00:44:42 2022] GET /identity/v3/projects/0da6bbbf30744afab58fab5feeb25970 => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:43.026631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:44:43.029708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:43.120862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561be6c5-2cb5-42b3-86f4-872bd122e195 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:44:43.153525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9ef584-3aa9-4dfd-9e18-0bbf2c3533ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:44:43.154110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef584-3aa9-4dfd-9e18-0bbf2c3533ab None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:44:43.154382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef584-3aa9-4dfd-9e18-0bbf2c3533ab None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:44:43.154651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9ef584-3aa9-4dfd-9e18-0bbf2c3533ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:44:43.155362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef584-3aa9-4dfd-9e18-0bbf2c3533ab None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=0da6bbbf30744afab58fab5feeb25970, user.id=bf72f22f0f574e04bc76fc9089e97ac4) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:44:43.155663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef584-3aa9-4dfd-9e18-0bbf2c3533ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:44:43.157092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9ef584-3aa9-4dfd-9e18-0bbf2c3533ab None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:44:43.162555 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 469/905] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:44:43 2022] GET /identity/v3/role_assignments?user.id=bf72f22f0f574e04bc76fc9089e97ac4&scope.project.id=0da6bbbf30744afab58fab5feeb25970&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:44:53.010880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=78 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:44:53.160088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=66 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:38.247170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a19e-a3cb-45ad-89c0-f636b466e20a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:38.247842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a19e-a3cb-45ad-89c0-f636b466e20a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:38.247842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a19e-a3cb-45ad-89c0-f636b466e20a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:38.249239 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 437/906] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:45:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:45:38.254822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8592a-4903-4ff4-9b0a-2460e2d20720 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:38.255303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8592a-4903-4ff4-9b0a-2460e2d20720 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:38.255405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8592a-4903-4ff4-9b0a-2460e2d20720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:38.278316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-81d8592a-4903-4ff4-9b0a-2460e2d20720 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:45:38.279535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-81d8592a-4903-4ff4-9b0a-2460e2d20720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:45:38.384338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 470/907] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:45:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:45:38.674018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a19e-a3cb-45ad-89c0-f636b466e20a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:38.799808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a19e-a3cb-45ad-89c0-f636b466e20a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:38.847397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:38.848165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:38.848493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:38.848757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:38.859934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:38.861456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:38.877693 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:38.878819 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:38.880424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:38.880424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:38.880424 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:38.885057 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 438/908] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:38.886160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8592a-4903-4ff4-9b0a-2460e2d20720 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:39.004870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8592a-4903-4ff4-9b0a-2460e2d20720 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:39.038941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:39.046149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae689b3a-763a-4446-9c5f-09cafe2a1ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:39.046790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae689b3a-763a-4446-9c5f-09cafe2a1ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:39.047068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae689b3a-763a-4446-9c5f-09cafe2a1ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:39.047341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae689b3a-763a-4446-9c5f-09cafe2a1ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:39.047985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae689b3a-763a-4446-9c5f-09cafe2a1ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:39.048274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae689b3a-763a-4446-9c5f-09cafe2a1ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:39.049667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae689b3a-763a-4446-9c5f-09cafe2a1ed1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:39.071442 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 471/909] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:39.077477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:39.143317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:39.206800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37098440-17c5-4e44-a325-9a8ac778d4d9 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:39.251878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e26fc-4cce-4b07-a67e-985764fe9a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:39.253205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e26fc-4cce-4b07-a67e-985764fe9a43 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:39.253659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e26fc-4cce-4b07-a67e-985764fe9a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:39.254650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e26fc-4cce-4b07-a67e-985764fe9a43 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:39.255511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e26fc-4cce-4b07-a67e-985764fe9a43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:39.257123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e26fc-4cce-4b07-a67e-985764fe9a43 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:39.292348 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 439/910] 10.222.0.39 () {60 vars in 1132 bytes} [Sun Oct 23 00:45:39 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:45:41.629610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adbca-db84-400c-9a91-432d5b48c138 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:41.630196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adbca-db84-400c-9a91-432d5b48c138 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:41.630298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627adbca-db84-400c-9a91-432d5b48c138 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:41.631839 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 472/911] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:45:41.638090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681fda9-4663-4226-89ff-6b7dc3f2aeca None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:41.638603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681fda9-4663-4226-89ff-6b7dc3f2aeca None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:41.638708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6681fda9-4663-4226-89ff-6b7dc3f2aeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:41.658139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6681fda9-4663-4226-89ff-6b7dc3f2aeca None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:45:41.659302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6681fda9-4663-4226-89ff-6b7dc3f2aeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:45:41.736953 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 440/912] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:45:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:45:42.015803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adbca-db84-400c-9a91-432d5b48c138 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:42.143923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627adbca-db84-400c-9a91-432d5b48c138 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:42.181714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:42.182353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:42.182672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:42.182868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:42.196467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:45:42.199554 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:42.200652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:42.201552 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 473/913] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:45:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:42.206979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681fda9-4663-4226-89ff-6b7dc3f2aeca None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:42.250586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:42.316161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6681fda9-4663-4226-89ff-6b7dc3f2aeca None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:42.358315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:42.359051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:42.359203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:42.359466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:42.360192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:42.360531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:42.361871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:42.371664 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 441/914] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:45:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:42.377460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:42.439812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:42.499028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745cdbf9-afce-464c-9189-ffca1b0a472a None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:42.539430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:42.540109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:42.540473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:42.540824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:42.552544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:42.554213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:42.570807 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:42.571829 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:42.572886 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:42.572886 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:42.572886 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:42.572886 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 474/915] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:45:42.577949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:42.690526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:42.694601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbd5c1-afbc-48c8-9eb7-c0c72507c1fe None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:42.735358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:42.736113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:42.736460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:42.736765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:42.737500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:42.737805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:42.739342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:42.752543 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 442/916] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:45:42.758979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:42.886538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514e7141-5d5e-4e01-bfd6-496fbc17bc50 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:42.888718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:42.927944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:42.928594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:42.928870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:42.929138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:42.936930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:42.938362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:45:42.950964 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:42.952223 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:42.953436 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:42.955049 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 475/917] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:42 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:42.960977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:43.074823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a453d01-20ea-4c46-b465-de81bd53625d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:43.108828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:43.109369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:43.109637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:43.109948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:43.110609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:43.110909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:43.112316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:43.132119 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 443/918] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Oct 23 00:45:42 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:43.138234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:43.245522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fa8b1-36c7-431e-9c4b-f6ef827275e1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:43.283834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:43.284497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:43.284780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:43.285052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:43.297690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:43.299405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:43.316880 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:43.318077 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:43.319215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:43.319215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:43.319215 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:43.323082 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 476/919] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:43.324581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:43.349652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:43.447580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166de842-7ad9-44b1-a1ab-706e04a03a43 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:43.496051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:43.496892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:43.497317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:43.497634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:43.498584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:43.498997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:43.500892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:43.520671 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 444/920] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:45:43.522531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:43.544948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:43.648199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fef8e-496b-4b66-a19b-fbbbb867ad69 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:43.692500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:43.693144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:43.693423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:43.693697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:43.700953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:43.702551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:45:43.716535 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:43.717613 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:43.718494 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:43.720454 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 477/921] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:45:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:43.725311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:43.818126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:43.851625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121907f3-158d-4503-a3bc-8204264f6e36 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:43.895571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e34659-a516-4c7f-8b03-60aee68ee768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:43.896326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34659-a516-4c7f-8b03-60aee68ee768 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:43.896725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34659-a516-4c7f-8b03-60aee68ee768 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:43.897064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e34659-a516-4c7f-8b03-60aee68ee768 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:43.898000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e34659-a516-4c7f-8b03-60aee68ee768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:43.898310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e34659-a516-4c7f-8b03-60aee68ee768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:43.899947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e34659-a516-4c7f-8b03-60aee68ee768 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:43.920015 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 445/922] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:45:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:45:43.921353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:44.010950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:44.045291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b806b33-e01f-4d3c-923a-3ffa633c0b81 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:44.093695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce07157-456b-4a65-a1d6-542bc1d3ea61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:44.094673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce07157-456b-4a65-a1d6-542bc1d3ea61 None admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:44.095134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce07157-456b-4a65-a1d6-542bc1d3ea61 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:44.095555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce07157-456b-4a65-a1d6-542bc1d3ea61 None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/145f39f828664c56903ae5aeffa4dd52/roles/d36d481771e04201bdf12458d1795e34` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:44.112792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce07157-456b-4a65-a1d6-542bc1d3ea61 None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=145f39f828664c56903ae5aeffa4dd52, role_id=d36d481771e04201bdf12458d1795e34)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:44.114290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce07157-456b-4a65-a1d6-542bc1d3ea61 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:44.128627 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 478/923] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:45:43 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/145f39f828664c56903ae5aeffa4dd52/roles/d36d481771e04201bdf12458d1795e34 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:45:46.299832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8fdc3-22a5-49cc-94ae-2159aac2a8de None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:46.300374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8fdc3-22a5-49cc-94ae-2159aac2a8de None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:46.300468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8fdc3-22a5-49cc-94ae-2159aac2a8de None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:46.301997 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 446/924] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:45:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:45:46.307057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e5169-4390-4788-9edf-f0cfa6246d46 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:46.307473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e5169-4390-4788-9edf-f0cfa6246d46 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:46.307599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e5169-4390-4788-9edf-f0cfa6246d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:46.326186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a33e5169-4390-4788-9edf-f0cfa6246d46 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:45:46.327552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a33e5169-4390-4788-9edf-f0cfa6246d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:45:46.397704 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 479/925] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:45:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:45:46.728362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8fdc3-22a5-49cc-94ae-2159aac2a8de None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:46.850442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8fdc3-22a5-49cc-94ae-2159aac2a8de None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:46.888671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:46.889298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:46.889570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:46.889836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:46.896802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:46.900624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:45:46.906116 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:46.907059 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:46.907946 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 447/926] 10.222.0.39 () {56 vars in 1102 bytes} [Sun Oct 23 00:45:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:46.914057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e5169-4390-4788-9edf-f0cfa6246d46 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:46.973722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:47.020684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e5169-4390-4788-9edf-f0cfa6246d46 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:47.056582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:47.057290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:47.057579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:47.057875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:47.058574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] RBAC: Adding query filter params (name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:47.058914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:47.060549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:47.070222 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 480/927] 10.222.0.39 () {56 vars in 1103 bytes} [Sun Oct 23 00:45:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:47.076656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:47.189485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b26932-289f-4bd0-a046-d5c4baeb2527 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:47.226824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:47.227509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:47.227837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:47.228076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:47.239141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:47.240682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:47.255736 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:47.256894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:47.258335 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:47.258335 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:47.258335 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:47.258335 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 448/928] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:47.263305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:47.368669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b166-57e8-4043-8ce3-2491773edf63 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:47.385648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:47.408526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:47.409224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:47.409575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:47.409927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:47.410577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:47.410878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:47.412227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:47.423747 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 481/929] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:47.430159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:47.483571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:47.543113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9f14d-f210-41ce-b4ac-b20685613b3d None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:47.583608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:47.584336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:47.584703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:47.585011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:47.593223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:47.594801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:45:47.606865 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:47.607873 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:47.608957 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 449/930] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:47.614792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:47.726272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eca4b6c-5b32-4edf-b80a-b96dbab7fbc1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:47.765026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:47.765659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:47.766012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:47.766361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:47.767279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:47.767614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:47.769296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:47.788169 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 482/931] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Oct 23 00:45:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:47.794480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:47.904358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644f0953-5857-44b2-a660-73148d1ab932 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:47.916032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:47.942355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:47.943053 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:47.943333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:47.943737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:48.012763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:48.014122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:48.122357 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 450/932] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:45:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:45:48.128905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:48.167511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:48.238879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56f1c5-14ac-42c2-a93d-c3f09517c704 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:48.276685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:48.277321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:48.277673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:48.277997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:48.290289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:48.291928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:48.308442 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:48.309649 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:48.310519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:48.310519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:48.310519 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:48.310519 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 483/933] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:48.315341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:48.327571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:48.421689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95bcbaa-8b45-41f6-9823-1bae4e1aaf87 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:48.458249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:48.458942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:48.459137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:48.459409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:48.460054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:48.460389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:48.461750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:48.473196 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 451/934] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:48.479121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:48.577415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:48.594500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a72693f-cb2e-4991-b688-18f360017e46 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:48.629866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:48.630525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:48.630811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:48.631082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:48.638527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:48.640027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:45:48.652198 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:48.653273 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:48.654283 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 484/935] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:48.659557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:48.733857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:48.769445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e833f-ac36-44aa-a1ba-f9afab5e2c02 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:48.805198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000c228-dca7-45cd-a5b8-ff0fd23e00a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:48.805860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c228-dca7-45cd-a5b8-ff0fd23e00a8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:48.806177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c228-dca7-45cd-a5b8-ff0fd23e00a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:48.806450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000c228-dca7-45cd-a5b8-ff0fd23e00a8 None admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:48.807189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000c228-dca7-45cd-a5b8-ff0fd23e00a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:48.807498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000c228-dca7-45cd-a5b8-ff0fd23e00a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:48.811185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000c228-dca7-45cd-a5b8-ff0fd23e00a8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:48.822959 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 452/936] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:48.830297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:48.933992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d02ba2-b4c9-4cdb-a64e-d01b6cb54871 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:48.967825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed60cf6-a7c4-412d-8f74-34f6a4ab2296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:48.968554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed60cf6-a7c4-412d-8f74-34f6a4ab2296 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:48.968840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed60cf6-a7c4-412d-8f74-34f6a4ab2296 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:48.969157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed60cf6-a7c4-412d-8f74-34f6a4ab2296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:48.969880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed60cf6-a7c4-412d-8f74-34f6a4ab2296 None admin] RBAC: Adding query filter params (role.id=d36d481771e04201bdf12458d1795e34, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=145f39f828664c56903ae5aeffa4dd52) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:48.970194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed60cf6-a7c4-412d-8f74-34f6a4ab2296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:48.971803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed60cf6-a7c4-412d-8f74-34f6a4ab2296 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:48.979623 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 485/937] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:45:48 2022] GET /identity/v3/role_assignments?user.id=145f39f828664c56903ae5aeffa4dd52&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=d36d481771e04201bdf12458d1795e34 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:51.266362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094eccc-ac19-425c-8577-ed3905d5935b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:51.266885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094eccc-ac19-425c-8577-ed3905d5935b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:51.266885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0094eccc-ac19-425c-8577-ed3905d5935b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:51.268343 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 453/938] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:45:51.274114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364d7db-3666-410c-8d07-1be1192676d0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:51.274654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364d7db-3666-410c-8d07-1be1192676d0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:51.274654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364d7db-3666-410c-8d07-1be1192676d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:51.293167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e364d7db-3666-410c-8d07-1be1192676d0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:45:51.294333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e364d7db-3666-410c-8d07-1be1192676d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:45:51.297179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:51.371317 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 486/939] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:45:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:45:51.650529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094eccc-ac19-425c-8577-ed3905d5935b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:51.760200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0094eccc-ac19-425c-8577-ed3905d5935b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:51.791188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:51.798303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:51.799037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:51.799350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:51.799702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:51.812215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:45:51.818199 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:51.819442 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:51.820454 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 454/940] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:45:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:51.825575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364d7db-3666-410c-8d07-1be1192676d0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:51.927942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364d7db-3666-410c-8d07-1be1192676d0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:51.966316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:51.966927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:51.967199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:51.967466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:51.968121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:51.968519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:51.970737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:51.978887 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 487/941] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:45:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:51.985253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:52.096599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de0f5c6-a1d6-4183-9b23-9cce2457d538 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:52.134893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:52.135526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:52.135888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:52.136151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:52.149171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:52.150838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:52.166476 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:52.167605 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:52.169134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:52.169134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:52.169134 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:52.169134 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 455/942] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:52.173917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:52.177324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:52.283106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28951bc5-1ab3-4e0c-89be-4bab899a97dd None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:52.317576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:52.318180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:52.318492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:52.318726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:52.319390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:52.319654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:52.321034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:52.332829 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 488/943] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:52.338832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:52.386750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:52.450079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c098b-7c10-4af4-a0a9-6b93b089a549 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:52.487903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:52.489265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:52.489663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:52.490110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:52.499810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:52.501446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:45:52.514978 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:52.515983 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:52.522469 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 456/944] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:45:52.523168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:52.591858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:52.634074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae07c6c-0957-4538-b284-1a88dc2d8314 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:52.671791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:52.672424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:52.672705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:52.672977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:52.673705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:52.674007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:52.675426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:52.697543 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 489/945] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Oct 23 00:45:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:52.698768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:52.804198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72c644-2ff6-40ca-98d9-701fef65ae1c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:52.806075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:52.842090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:52.842667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:52.842942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:52.843234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:52.856062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:52.857828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:52.873789 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:52.874986 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:52.875972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:52.875972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:52.875972 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:52.875972 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 457/946] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:52.881304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:52.997051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8621fb7-8c9f-488f-9f3b-d09baff962d1 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:53.020268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:53.036531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:53.037200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:53.037470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:53.037809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:53.038542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:53.038849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:53.040368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:53.052669 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 490/947] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:53.059405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:53.168836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c739a-a496-420b-a8d6-a37738b29077 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:53.204318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:53.204944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:53.205221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:53.205507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:53.213445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:53.214863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:45:53.225821 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:53.226747 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:53.227668 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 458/948] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:45:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:45:53.233101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:53.345498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db0a08d-6bb5-4067-8106-8750e93ead7b None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:53.382517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e67279-c683-467c-9ad0-582d071bdc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:53.383423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67279-c683-467c-9ad0-582d071bdc8c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:53.383702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67279-c683-467c-9ad0-582d071bdc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:53.383975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e67279-c683-467c-9ad0-582d071bdc8c None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:53.384698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67279-c683-467c-9ad0-582d071bdc8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:53.385000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67279-c683-467c-9ad0-582d071bdc8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:53.386416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e67279-c683-467c-9ad0-582d071bdc8c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:53.399522 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 491/949] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:45:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:53.405832 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:53.423791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:53.528514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ec81e4-a440-4d3d-8621-1a1b1b494eed None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:53.576845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7aca2-309e-47c3-8f8a-4523dd1d0aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:53.577502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7aca2-309e-47c3-8f8a-4523dd1d0aee None admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:53.577774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7aca2-309e-47c3-8f8a-4523dd1d0aee None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:53.578045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7aca2-309e-47c3-8f8a-4523dd1d0aee None admin] PATH_INFO: `/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/145f39f828664c56903ae5aeffa4dd52/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:53.579939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7aca2-309e-47c3-8f8a-4523dd1d0aee None admin] RBAC: Authorizing `identity:create_grant(project_id=b5fa3bce9d5a404bb7352ae8867cf4ab, user_id=145f39f828664c56903ae5aeffa4dd52, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:53.581394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7aca2-309e-47c3-8f8a-4523dd1d0aee None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:53.594682 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 459/950] 10.222.0.39 () {58 vars in 1439 bytes} [Sun Oct 23 00:45:53 2022] PUT /identity/v3/projects/b5fa3bce9d5a404bb7352ae8867cf4ab/users/145f39f828664c56903ae5aeffa4dd52/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:45:55.783761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6de94c-a87a-465d-807a-ff41c48f9896 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:55.784362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6de94c-a87a-465d-807a-ff41c48f9896 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:55.784362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6de94c-a87a-465d-807a-ff41c48f9896 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:55.788331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 492/951] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:45:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:45:55.792526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902f30a-c132-4492-9661-d38bd4b3c3ca None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:55.792925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902f30a-c132-4492-9661-d38bd4b3c3ca None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:55.793079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f902f30a-c132-4492-9661-d38bd4b3c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:55.812134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f902f30a-c132-4492-9661-d38bd4b3c3ca None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:45:55.813319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f902f30a-c132-4492-9661-d38bd4b3c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:45:55.887406 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 460/952] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:45:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:45:56.202344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6de94c-a87a-465d-807a-ff41c48f9896 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:56.249419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:56.312280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6de94c-a87a-465d-807a-ff41c48f9896 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:56.348163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:56.348938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:56.349202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:56.349463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:56.360079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 00:45:56.365547 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:56.366680 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:56.367516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 493/953] 10.222.0.39 () {56 vars in 1096 bytes} [Sun Oct 23 00:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:56.372874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902f30a-c132-4492-9661-d38bd4b3c3ca None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:56.375894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:56.475395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f902f30a-c132-4492-9661-d38bd4b3c3ca None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:56.515129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:56.516005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:56.516321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:56.516678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:56.517546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:56.517882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:56.519709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:56.529890 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 461/954] 10.222.0.39 () {56 vars in 1099 bytes} [Sun Oct 23 00:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:56.536671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:56.642417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:56.647528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177106b-b4bb-40f3-b60d-18f9fbacb187 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:56.686156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:56.686936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:56.687217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:56.687487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:56.698607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:56.700325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:56.717317 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:56.724229 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:56.725392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:56.725392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:56.725392 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:56.725392 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 494/955] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 00:45:56.725740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:56.797109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:56.833615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a795942-4bf5-43f7-8bd3-10acabf0d4a5 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:56.870265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:56.870941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:56.871257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:56.871542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:56.872321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:56.872727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:56.874753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:56.887805 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 462/956] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:56.894051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:57.003452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3413-b5e8-4b62-97b5-4f6e56743dae None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:57.040277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:57.040942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:57.041222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:57.041494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:57.049341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:57.050874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 00:45:57.063001 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:57.064122 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:45:57.065139 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 495/957] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:57.070453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:57.174624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9cb6b-b185-4655-933f-374b2673f46c None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:57.201332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:57.211876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:57.212536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:57.212816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:57.213086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:57.213830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:57.214133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:57.215616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:57.234841 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 463/958] 10.222.0.39 () {56 vars in 1143 bytes} [Sun Oct 23 00:45:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:57.241086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:57.257129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:57.348243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f871dd5-a96b-42e2-ac7c-5965f748ef52 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:57.381435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:57.382037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:57.382314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:57.382585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:57.454133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:57.455583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:57.470329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:57.563684 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 496/959] 10.222.0.39 () {58 vars in 1282 bytes} [Sun Oct 23 00:45:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 00:45:57.570298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:57.669901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d3154c-31d1-4503-9a68-3aa7fe10f66f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:57.702095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:57.702708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:57.702977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:57.703237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:57.713329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:57.714735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 00:45:57.728532 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:57.729610 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 00:45:57.730702 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 00:45:57.730702 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 00:45:57.730702 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:57.730702 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 464/960] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:45:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:57.735421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:57.830214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f15951-6a87-4820-92fc-1bddd77fe501 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:57.842761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:57.862779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:57.863964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:57.864261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:57.864550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:57.865187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:57.865487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:57.866835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:57.877761 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 497/961] 10.222.0.39 () {56 vars in 1109 bytes} [Sun Oct 23 00:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:57.884202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:57.917341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:57.978557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7005032f-8dc9-4199-b2a3-32ba22e2906f None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:58.011219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:58.011825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:58.012101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:58.012397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:58.018681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:58.020125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 00:45:58.030881 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application with Lock( Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:45:58.031894 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: ERROR keystone.server.flask.application Oct 23 00:45:58.037452 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 465/962] 10.222.0.39 () {56 vars in 1111 bytes} [Sun Oct 23 00:45:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:45:58.038069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:58.133660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2579e-635e-4b08-8cca-480af4086992 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:58.165371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3c739c-04ca-45e2-9783-c3c497a49644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:58.165960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c739c-04ca-45e2-9783-c3c497a49644 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:58.166238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c739c-04ca-45e2-9783-c3c497a49644 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:58.166509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3c739c-04ca-45e2-9783-c3c497a49644 None admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:58.167195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c739c-04ca-45e2-9783-c3c497a49644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:58.167495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c739c-04ca-45e2-9783-c3c497a49644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:58.168929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3c739c-04ca-45e2-9783-c3c497a49644 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:58.180686 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 498/963] 10.222.0.39 () {56 vars in 1148 bytes} [Sun Oct 23 00:45:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:45:58.186554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:45:58.275653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:45:58.281477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb0122c-fa31-476c-9dc5-82cd3ba09449 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:45:58.313802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196efd-6805-46d3-863c-f0badaf33a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:45:58.314383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196efd-6805-46d3-863c-f0badaf33a6f None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:45:58.314660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196efd-6805-46d3-863c-f0badaf33a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:45:58.314928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196efd-6805-46d3-863c-f0badaf33a6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:45:58.315638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196efd-6805-46d3-863c-f0badaf33a6f None admin] RBAC: Adding query filter params (role.id=4ea11f7df83546718222b4c38fd2c9f2, scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab, user.id=145f39f828664c56903ae5aeffa4dd52) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:45:58.315944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196efd-6805-46d3-863c-f0badaf33a6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:45:58.317395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196efd-6805-46d3-863c-f0badaf33a6f None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:45:58.322918 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 466/964] 10.222.0.39 () {56 vars in 1374 bytes} [Sun Oct 23 00:45:58 2022] GET /identity/v3/role_assignments?user.id=145f39f828664c56903ae5aeffa4dd52&scope.project.id=b5fa3bce9d5a404bb7352ae8867cf4ab&role.id=4ea11f7df83546718222b4c38fd2c9f2 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:46:00.444276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade799-5434-4e99-8c08-ee8c0044466c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:00.444852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade799-5434-4e99-8c08-ee8c0044466c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:00.444852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ade799-5434-4e99-8c08-ee8c0044466c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:00.446199 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 499/965] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:46:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:46:00.451384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578ce2b-85c6-4723-bf4e-51be31b3ca13 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:00.451769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578ce2b-85c6-4723-bf4e-51be31b3ca13 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:00.451931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578ce2b-85c6-4723-bf4e-51be31b3ca13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:00.470068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b578ce2b-85c6-4723-bf4e-51be31b3ca13 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:46:00.471229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b578ce2b-85c6-4723-bf4e-51be31b3ca13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:46:00.545070 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 467/966] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:46:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:46:00.815738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade799-5434-4e99-8c08-ee8c0044466c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:00.846116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:00.914590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ade799-5434-4e99-8c08-ee8c0044466c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:00.948008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:00.948657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:00.948939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:00.949213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:00.949784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:00.951133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:46:00.959671 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:46:00.960980 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:00.962111 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 500/967] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:46:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:46:00.966804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578ce2b-85c6-4723-bf4e-51be31b3ca13 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:01.052661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:01.062253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578ce2b-85c6-4723-bf4e-51be31b3ca13 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:01.094393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7040519-5ce3-43af-9dc5-46324e52bd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:01.094927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7040519-5ce3-43af-9dc5-46324e52bd08 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:01.095203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7040519-5ce3-43af-9dc5-46324e52bd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:01.095472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7040519-5ce3-43af-9dc5-46324e52bd08 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:01.096088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7040519-5ce3-43af-9dc5-46324e52bd08 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:46:01.096411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7040519-5ce3-43af-9dc5-46324e52bd08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:01.097778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7040519-5ce3-43af-9dc5-46324e52bd08 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:01.102359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 468/968] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Oct 23 00:46:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:46:01.107937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:01.198333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:01.204187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63de477-e97e-486f-995e-d645c5a19f17 None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:01.235894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b9881-7a6a-43c2-87aa-02007e948c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:01.236504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b9881-7a6a-43c2-87aa-02007e948c4c None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:01.236781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b9881-7a6a-43c2-87aa-02007e948c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:01.237048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b9881-7a6a-43c2-87aa-02007e948c4c None admin] PATH_INFO: `/v3/services` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:01.237659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b9881-7a6a-43c2-87aa-02007e948c4c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:46:01.238527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b9881-7a6a-43c2-87aa-02007e948c4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:01.239860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b9881-7a6a-43c2-87aa-02007e948c4c None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:01.244023 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 501/969] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Oct 23 00:46:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:46:03.341177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36302fa6-6ace-48c5-b262-6491da1370c1 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:03.341739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36302fa6-6ace-48c5-b262-6491da1370c1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:03.341739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36302fa6-6ace-48c5-b262-6491da1370c1 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:03.343057 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 469/970] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:46:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:46:03.348211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e264-d79a-4bfe-8364-3b2c8db99e44 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:03.348700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e264-d79a-4bfe-8364-3b2c8db99e44 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:03.348799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5e264-d79a-4bfe-8364-3b2c8db99e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:03.366915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-98e5e264-d79a-4bfe-8364-3b2c8db99e44 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:46:03.368088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-98e5e264-d79a-4bfe-8364-3b2c8db99e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:46:03.432726 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 502/971] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:46:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:46:03.699361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36302fa6-6ace-48c5-b262-6491da1370c1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:03.799427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36302fa6-6ace-48c5-b262-6491da1370c1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:03.831935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4002c-af1c-48c8-a449-baffc996cf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:03.832563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4002c-af1c-48c8-a449-baffc996cf25 None admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:03.832843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4002c-af1c-48c8-a449-baffc996cf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:03.833112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4002c-af1c-48c8-a449-baffc996cf25 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:03.833754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4002c-af1c-48c8-a449-baffc996cf25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:03.835091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4002c-af1c-48c8-a449-baffc996cf25 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:03.849512 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 470/972] 10.222.0.39 () {60 vars in 1141 bytes} [Sun Oct 23 00:46:03 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:46:05.973348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bf1b3-fb66-4a53-84a9-8b746b070838 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:05.973881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bf1b3-fb66-4a53-84a9-8b746b070838 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:05.973881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bf1b3-fb66-4a53-84a9-8b746b070838 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:05.975220 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 503/973] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:46:05.980363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee67222-455e-414e-ad61-fffed037a9b4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:05.980794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee67222-455e-414e-ad61-fffed037a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:05.980918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee67222-455e-414e-ad61-fffed037a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:05.998727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8ee67222-455e-414e-ad61-fffed037a9b4 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:46:05.999892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8ee67222-455e-414e-ad61-fffed037a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:46:06.009588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:06.071514 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 471/974] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:46:05 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 00:46:06.337611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bf1b3-fb66-4a53-84a9-8b746b070838 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:06.423889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:06.435531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bf1b3-fb66-4a53-84a9-8b746b070838 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:06.467198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:06.467810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:06.468086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:06.468378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] PATH_INFO: `/v3/services/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:06.468946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:06.470361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:46:06.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:46:06.480526 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:06.481532 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 504/975] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:46:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:46:06.486331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee67222-455e-414e-ad61-fffed037a9b4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:06.593746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee67222-455e-414e-ad61-fffed037a9b4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:06.628042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:06.628778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:06.629059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:06.629330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:06.629953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:46:06.630253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:06.631603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:06.636835 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 472/976] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Oct 23 00:46:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:46:06.642888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:06.739741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b01603-b410-4421-ad0e-d6fe0b28860f None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:06.772401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357985c-76bc-41af-acf4-7f9a77a29b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:06.772993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357985c-76bc-41af-acf4-7f9a77a29b22 None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:06.773273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357985c-76bc-41af-acf4-7f9a77a29b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:06.773545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357985c-76bc-41af-acf4-7f9a77a29b22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:06.774242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357985c-76bc-41af-acf4-7f9a77a29b22 None admin] RBAC: Adding query filter params (interface=public, service_id=67dd766c13ee4c0f9baf0035d881a16c, region_id=RegionOne) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:46:06.774548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357985c-76bc-41af-acf4-7f9a77a29b22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:06.775894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357985c-76bc-41af-acf4-7f9a77a29b22 None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:06.780749 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 505/977] 10.222.0.39 () {56 vars in 1251 bytes} [Sun Oct 23 00:46:06 2022] GET /identity/v3/endpoints?service_id=67dd766c13ee4c0f9baf0035d881a16c&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:46:06.786822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:06.790445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:06.882149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7fd4-a0a8-4c00-b192-29788614b4c2 None admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:06.915509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13427972-2333-4541-b2dc-a110a0a8f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:06.916108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427972-2333-4541-b2dc-a110a0a8f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:06.916417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427972-2333-4541-b2dc-a110a0a8f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:06.916690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13427972-2333-4541-b2dc-a110a0a8f4c8 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:06.917288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13427972-2333-4541-b2dc-a110a0a8f4c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:06.918637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13427972-2333-4541-b2dc-a110a0a8f4c8 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:06.925256 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 473/978] 10.222.0.39 () {56 vars in 1087 bytes} [Sun Oct 23 00:46:06 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:46:09.034679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb09e-8487-4fb2-a643-1224ce2058d9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:09.035237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb09e-8487-4fb2-a643-1224ce2058d9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:09.035237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0bb09e-8487-4fb2-a643-1224ce2058d9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:09.036632 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 506/979] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:46:09.041830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775ab7-4cd9-4b7f-b8a6-e6186413d509 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:09.042269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775ab7-4cd9-4b7f-b8a6-e6186413d509 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:09.042379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775ab7-4cd9-4b7f-b8a6-e6186413d509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:09.060425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-51775ab7-4cd9-4b7f-b8a6-e6186413d509 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:46:09.061596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-51775ab7-4cd9-4b7f-b8a6-e6186413d509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:46:09.119985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:09.125651 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 474/980] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:46:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 00:46:09.392039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb09e-8487-4fb2-a643-1224ce2058d9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:09.411064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:09.489697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0bb09e-8487-4fb2-a643-1224ce2058d9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:09.527685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:09.528405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:09.528716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:09.528984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] PATH_INFO: `/v3/services/placement` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:09.529639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:09.531112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 00:46:09.541644 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 00:46:09.542652 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application with Lock( Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self._enter() Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application return self.creator() Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application created_value = creator( Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: ERROR keystone.server.flask.application Oct 23 00:46:09.543501 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 507/981] 10.222.0.39 () {56 vars in 1117 bytes} [Sun Oct 23 00:46:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 00:46:09.548919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51775ab7-4cd9-4b7f-b8a6-e6186413d509 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:09.646111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51775ab7-4cd9-4b7f-b8a6-e6186413d509 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:09.678828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe93e1-a599-4bf4-83cb-eb093237b4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:09.679423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe93e1-a599-4bf4-83cb-eb093237b4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:09.679698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe93e1-a599-4bf4-83cb-eb093237b4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:09.679989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe93e1-a599-4bf4-83cb-eb093237b4c6 None admin] PATH_INFO: `/v3/services` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:09.680632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe93e1-a599-4bf4-83cb-eb093237b4c6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:46:09.680931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe93e1-a599-4bf4-83cb-eb093237b4c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:09.682273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe93e1-a599-4bf4-83cb-eb093237b4c6 None admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:09.687159 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 475/982] 10.222.0.39 () {56 vars in 1116 bytes} [Sun Oct 23 00:46:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:46:09.692946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:46:09.771598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:09.789692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41bfd1b-c6bd-4282-b20e-85b480e496be None admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:46:09.821356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac3bf4-0341-4d1a-bbf2-bc2d62a3af6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:46:09.821959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac3bf4-0341-4d1a-bbf2-bc2d62a3af6d None admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:46:09.822238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac3bf4-0341-4d1a-bbf2-bc2d62a3af6d None admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:46:09.822510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac3bf4-0341-4d1a-bbf2-bc2d62a3af6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:46:09.823180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac3bf4-0341-4d1a-bbf2-bc2d62a3af6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:46:09.824566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac3bf4-0341-4d1a-bbf2-bc2d62a3af6d None admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:46:09.843669 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 508/983] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:46:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:46:19.684373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=69 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:46:19.839803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:13.449912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ade2c-63fd-4335-ab67-9f11d16ba5cd None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:13.450353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ade2c-63fd-4335-ab67-9f11d16ba5cd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:13.450559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ade2c-63fd-4335-ab67-9f11d16ba5cd None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:13.452049 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 476/984] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:13.457614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a04b6-b199-4690-be6b-d88a9efe1977 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:13.458034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a04b6-b199-4690-be6b-d88a9efe1977 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:13.458170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56a04b6-b199-4690-be6b-d88a9efe1977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:13.581497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b56a04b6-b199-4690-be6b-d88a9efe1977 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:13.583917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b56a04b6-b199-4690-be6b-d88a9efe1977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:13.717274 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 509/985] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:14.017191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bb735-64c2-4964-9c19-10dbf43a7c40 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:14.017482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bb735-64c2-4964-9c19-10dbf43a7c40 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:14.017914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bb735-64c2-4964-9c19-10dbf43a7c40 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:14.020193 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 477/986] 10.222.0.39 () {54 vars in 964 bytes} [Sun Oct 23 00:47:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:14.026674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba137-e55b-4c51-9e58-b0f69415f8a0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:14.026981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba137-e55b-4c51-9e58-b0f69415f8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:14.027238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eba137-e55b-4c51-9e58-b0f69415f8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:14.112810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-30eba137-e55b-4c51-9e58-b0f69415f8a0 None None] MFA Rules not processed for user `bb5f82d3cfc14eacbe4cc40bcd0dfb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:14.114014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-30eba137-e55b-4c51-9e58-b0f69415f8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:14.136141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:14.193946 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 510/987] 10.222.0.39 () {58 vars in 1061 bytes} [Sun Oct 23 00:47:14 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:14.202731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bb735-64c2-4964-9c19-10dbf43a7c40 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:14.204527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-034bb735-64c2-4964-9c19-10dbf43a7c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:14.320085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bb735-64c2-4964-9c19-10dbf43a7c40 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:14.399137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e0c0b-537a-4c22-b82a-f4d90fa2b5e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:14.399638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0c0b-537a-4c22-b82a-f4d90fa2b5e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:14.399917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0c0b-537a-4c22-b82a-f4d90fa2b5e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:14.400187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e0c0b-537a-4c22-b82a-f4d90fa2b5e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:14.401867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f10e0c0b-537a-4c22-b82a-f4d90fa2b5e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:14.461337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:14.523661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e0c0b-537a-4c22-b82a-f4d90fa2b5e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:14.525139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10e0c0b-537a-4c22-b82a-f4d90fa2b5e6 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:14.545446 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 478/988] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:17.042812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bc3c-ff7c-412b-8c4e-afdb630ecea8 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:17.043246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bc3c-ff7c-412b-8c4e-afdb630ecea8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:17.043506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355bc3c-ff7c-412b-8c4e-afdb630ecea8 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:17.045190 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 511/989] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:17.051050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cfdc8-503e-4eff-bfa7-569677b87c68 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:17.051448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cfdc8-503e-4eff-bfa7-569677b87c68 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:17.051756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5cfdc8-503e-4eff-bfa7-569677b87c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:17.130493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ce5cfdc8-503e-4eff-bfa7-569677b87c68 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:17.131685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ce5cfdc8-503e-4eff-bfa7-569677b87c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:17.227609 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 479/990] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:17.512393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bc3c-ff7c-412b-8c4e-afdb630ecea8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:17.513908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5355bc3c-ff7c-412b-8c4e-afdb630ecea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:17.616111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355bc3c-ff7c-412b-8c4e-afdb630ecea8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:17.685156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:17.687515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52144cd5-5b9f-4c81-a256-db579350aabc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:17.688365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52144cd5-5b9f-4c81-a256-db579350aabc service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:17.688691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52144cd5-5b9f-4c81-a256-db579350aabc service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:17.689116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52144cd5-5b9f-4c81-a256-db579350aabc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:17.691436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-52144cd5-5b9f-4c81-a256-db579350aabc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:17.795997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52144cd5-5b9f-4c81-a256-db579350aabc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:17.797382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52144cd5-5b9f-4c81-a256-db579350aabc service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:17.814761 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 512/991] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:20.254800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1a0a-6342-40c6-8781-5346f4359d8f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:20.255297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1a0a-6342-40c6-8781-5346f4359d8f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:20.255297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e1a0a-6342-40c6-8781-5346f4359d8f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:20.256634 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 480/992] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:20.262298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8c39b-1ce3-4cf9-b24f-1f1796ffc8a2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:20.262695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8c39b-1ce3-4cf9-b24f-1f1796ffc8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:20.262843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8c39b-1ce3-4cf9-b24f-1f1796ffc8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:20.327919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0aa8c39b-1ce3-4cf9-b24f-1f1796ffc8a2 None None] MFA Rules not processed for user `0b5a204a6bc6400198969e6cf752082a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:20.329151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0aa8c39b-1ce3-4cf9-b24f-1f1796ffc8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:20.380207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:20.423975 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 513/993] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:20.708046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e1a0a-6342-40c6-8781-5346f4359d8f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:20.709575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0d2e1a0a-6342-40c6-8781-5346f4359d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:20.719074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:20.810755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e1a0a-6342-40c6-8781-5346f4359d8f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:20.882337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933914ea-6ea8-41dd-abcd-c4719540691b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:20.882931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933914ea-6ea8-41dd-abcd-c4719540691b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:20.883205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933914ea-6ea8-41dd-abcd-c4719540691b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:20.883470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933914ea-6ea8-41dd-abcd-c4719540691b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:20.885286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-933914ea-6ea8-41dd-abcd-c4719540691b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:20.944604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:20.989477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933914ea-6ea8-41dd-abcd-c4719540691b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:20.990974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933914ea-6ea8-41dd-abcd-c4719540691b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:21.009372 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 481/994] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:23.663181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104e86e-2bd9-4609-a23f-b053e5a4926e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:23.663719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104e86e-2bd9-4609-a23f-b053e5a4926e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:23.663719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b104e86e-2bd9-4609-a23f-b053e5a4926e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:23.665107 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 514/995] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:23.670260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de37ceb-4b7b-4d9d-b8e6-60e92484cce2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:23.670688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de37ceb-4b7b-4d9d-b8e6-60e92484cce2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:23.670803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de37ceb-4b7b-4d9d-b8e6-60e92484cce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:23.734570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7de37ceb-4b7b-4d9d-b8e6-60e92484cce2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:23.735746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7de37ceb-4b7b-4d9d-b8e6-60e92484cce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:23.828505 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 482/996] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:24.117949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104e86e-2bd9-4609-a23f-b053e5a4926e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:24.119472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b104e86e-2bd9-4609-a23f-b053e5a4926e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:24.218570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b104e86e-2bd9-4609-a23f-b053e5a4926e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:24.290208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd014e-c0aa-4c9c-bab1-cc7eab1ab6b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:24.290805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd014e-c0aa-4c9c-bab1-cc7eab1ab6b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:24.291083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd014e-c0aa-4c9c-bab1-cc7eab1ab6b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:24.291352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd014e-c0aa-4c9c-bab1-cc7eab1ab6b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:24.292931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-10dd014e-c0aa-4c9c-bab1-cc7eab1ab6b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:24.300051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:24.399005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd014e-c0aa-4c9c-bab1-cc7eab1ab6b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:24.400396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd014e-c0aa-4c9c-bab1-cc7eab1ab6b5 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:24.418187 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 515/997] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:24 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:27.206690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c7d1-9cc2-4183-8abf-c00596deb671 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:27.207217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c7d1-9cc2-4183-8abf-c00596deb671 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:27.207217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356c7d1-9cc2-4183-8abf-c00596deb671 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:27.208712 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 483/998] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:27.214349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa710a-cabe-46cc-8a7b-635615a19e7c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:27.214901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa710a-cabe-46cc-8a7b-635615a19e7c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:27.214901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fa710a-cabe-46cc-8a7b-635615a19e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:27.287856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-99fa710a-cabe-46cc-8a7b-635615a19e7c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:27.289115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-99fa710a-cabe-46cc-8a7b-635615a19e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:27.349268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:27.389211 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 516/999] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:27.689304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03249c3b-bfa9-44c9-923c-15740512277f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:27.690242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03249c3b-bfa9-44c9-923c-15740512277f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:27.690583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03249c3b-bfa9-44c9-923c-15740512277f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:27.691995 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 484/1000] 10.222.0.39 () {54 vars in 964 bytes} [Sun Oct 23 00:47:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:27.697642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ec74-6dd4-4d88-b37c-8648633c2eb6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:27.697925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ec74-6dd4-4d88-b37c-8648633c2eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:27.698197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ec74-6dd4-4d88-b37c-8648633c2eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:27.774603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d2b9ec74-6dd4-4d88-b37c-8648633c2eb6 None None] MFA Rules not processed for user `bb5f82d3cfc14eacbe4cc40bcd0dfb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:27.775796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d2b9ec74-6dd4-4d88-b37c-8648633c2eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:27.852984 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 517/1001] 10.222.0.39 () {58 vars in 1061 bytes} [Sun Oct 23 00:47:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:27.862274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03249c3b-bfa9-44c9-923c-15740512277f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:27.863754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-03249c3b-bfa9-44c9-923c-15740512277f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:27.883511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:27.968328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03249c3b-bfa9-44c9-923c-15740512277f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:28.041240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9dd8e-18ab-4d17-9660-eb9663c3077b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:28.041842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9dd8e-18ab-4d17-9660-eb9663c3077b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:28.042119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9dd8e-18ab-4d17-9660-eb9663c3077b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:28.042413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9dd8e-18ab-4d17-9660-eb9663c3077b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:28.043972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8fd9dd8e-18ab-4d17-9660-eb9663c3077b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:28.113633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:28.152276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9dd8e-18ab-4d17-9660-eb9663c3077b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:28.153717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd9dd8e-18ab-4d17-9660-eb9663c3077b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:28.172129 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 485/1002] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:30.604701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7bf95-ed01-4377-ab67-d0516426b32e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:30.605226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7bf95-ed01-4377-ab67-d0516426b32e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:30.605226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e7bf95-ed01-4377-ab67-d0516426b32e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:30.606504 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 518/1003] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:30.611879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b77621-42be-4351-b58b-6f872131c49c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:30.612332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b77621-42be-4351-b58b-6f872131c49c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:30.612448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b77621-42be-4351-b58b-6f872131c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:30.677884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c8b77621-42be-4351-b58b-6f872131c49c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:30.679065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c8b77621-42be-4351-b58b-6f872131c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:30.777319 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 486/1004] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:31.061284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7bf95-ed01-4377-ab67-d0516426b32e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:31.062855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a1e7bf95-ed01-4377-ab67-d0516426b32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:31.122611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:31.167151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e7bf95-ed01-4377-ab67-d0516426b32e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:31.239554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0b5191-acf9-4dca-a225-87b48d7ecbc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:31.240162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5191-acf9-4dca-a225-87b48d7ecbc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:31.240482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5191-acf9-4dca-a225-87b48d7ecbc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:31.240758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0b5191-acf9-4dca-a225-87b48d7ecbc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:31.242377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6b0b5191-acf9-4dca-a225-87b48d7ecbc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:31.350959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:31.353515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b5191-acf9-4dca-a225-87b48d7ecbc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:31.355006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0b5191-acf9-4dca-a225-87b48d7ecbc1 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:31.376642 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 519/1005] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:33.605392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6e129-a7a4-40b4-a68f-3e47caf1f01c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:33.605901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6e129-a7a4-40b4-a68f-3e47caf1f01c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:33.606011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6e129-a7a4-40b4-a68f-3e47caf1f01c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:33.607282 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 487/1006] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:33.612854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc0c75-d828-4de8-ae2f-2e6384e9a609 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:33.613319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc0c75-d828-4de8-ae2f-2e6384e9a609 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:33.613409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc0c75-d828-4de8-ae2f-2e6384e9a609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:33.682144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f3fc0c75-d828-4de8-ae2f-2e6384e9a609 None None] MFA Rules not processed for user `0b5a204a6bc6400198969e6cf752082a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:33.683331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f3fc0c75-d828-4de8-ae2f-2e6384e9a609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:33.778798 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 520/1007] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:33 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:34.060050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6e129-a7a4-40b4-a68f-3e47caf1f01c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:34.061614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7a6e129-a7a4-40b4-a68f-3e47caf1f01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:34.092343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:34.175375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6e129-a7a4-40b4-a68f-3e47caf1f01c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:34.247940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c804103-cece-4617-952b-4f473fb11af0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:34.248587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c804103-cece-4617-952b-4f473fb11af0 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:34.248867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c804103-cece-4617-952b-4f473fb11af0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:34.249137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c804103-cece-4617-952b-4f473fb11af0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:34.250723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4c804103-cece-4617-952b-4f473fb11af0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:34.331645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:34.370196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c804103-cece-4617-952b-4f473fb11af0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:34.371714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c804103-cece-4617-952b-4f473fb11af0 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:34.391691 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 488/1008] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:36.936698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9173-b2f9-472b-8bda-7f2f86ba7004 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:36.937247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9173-b2f9-472b-8bda-7f2f86ba7004 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:36.937247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d9173-b2f9-472b-8bda-7f2f86ba7004 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:36.938595 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 521/1009] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:36.944266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138ccbe-950c-4c89-939f-2e8bfaf3bbff None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:36.944728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138ccbe-950c-4c89-939f-2e8bfaf3bbff None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:36.944833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138ccbe-950c-4c89-939f-2e8bfaf3bbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:37.009336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b138ccbe-950c-4c89-939f-2e8bfaf3bbff None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:37.010516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b138ccbe-950c-4c89-939f-2e8bfaf3bbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:37.102914 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 489/1010] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:37.382925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d9173-b2f9-472b-8bda-7f2f86ba7004 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:37.384725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-170d9173-b2f9-472b-8bda-7f2f86ba7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:37.452795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:37.485894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d9173-b2f9-472b-8bda-7f2f86ba7004 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:37.557014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324d4ad-74be-46d9-a006-1fbdc401e54a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:37.557610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324d4ad-74be-46d9-a006-1fbdc401e54a service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:37.557884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324d4ad-74be-46d9-a006-1fbdc401e54a service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:37.558152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324d4ad-74be-46d9-a006-1fbdc401e54a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:37.559693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c324d4ad-74be-46d9-a006-1fbdc401e54a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:37.664751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324d4ad-74be-46d9-a006-1fbdc401e54a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:37.666092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324d4ad-74be-46d9-a006-1fbdc401e54a service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:37.676012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:37.684186 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 522/1011] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:39.848413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989779-42fa-4dce-9ad4-1a7576aabd3c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:39.848927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989779-42fa-4dce-9ad4-1a7576aabd3c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:39.849028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05989779-42fa-4dce-9ad4-1a7576aabd3c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:39.850297 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 490/1012] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:39.855675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0bdc7-3955-4d75-af06-9bb84b60007a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:39.856106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0bdc7-3955-4d75-af06-9bb84b60007a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:39.856216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced0bdc7-3955-4d75-af06-9bb84b60007a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:39.919805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ced0bdc7-3955-4d75-af06-9bb84b60007a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:39.921016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ced0bdc7-3955-4d75-af06-9bb84b60007a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:40.018555 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 523/1013] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:40.315572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989779-42fa-4dce-9ad4-1a7576aabd3c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:40.317144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-05989779-42fa-4dce-9ad4-1a7576aabd3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:40.356407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:40.418439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05989779-42fa-4dce-9ad4-1a7576aabd3c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:40.489359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13604e-81b0-4153-9bd7-5b06a765f642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:40.489954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13604e-81b0-4153-9bd7-5b06a765f642 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:40.490229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13604e-81b0-4153-9bd7-5b06a765f642 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:40.490495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13604e-81b0-4153-9bd7-5b06a765f642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:40.492114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1f13604e-81b0-4153-9bd7-5b06a765f642 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:40.573100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:40.598693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13604e-81b0-4153-9bd7-5b06a765f642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:40.600110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13604e-81b0-4153-9bd7-5b06a765f642 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:40.617349 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 491/1014] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:43.076984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22277d-9e4d-4a9a-9efe-9fb65a3ad9e3 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:43.077509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22277d-9e4d-4a9a-9efe-9fb65a3ad9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:43.077509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b22277d-9e4d-4a9a-9efe-9fb65a3ad9e3 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:43.078849 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 524/1015] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:43.084528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721d88e-d93c-4683-9a48-f0a37dbc8c1f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:43.084937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721d88e-d93c-4683-9a48-f0a37dbc8c1f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:43.085029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2721d88e-d93c-4683-9a48-f0a37dbc8c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:43.149848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2721d88e-d93c-4683-9a48-f0a37dbc8c1f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:43.151025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2721d88e-d93c-4683-9a48-f0a37dbc8c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:43.243101 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 492/1016] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:43.527297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22277d-9e4d-4a9a-9efe-9fb65a3ad9e3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:43.528857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5b22277d-9e4d-4a9a-9efe-9fb65a3ad9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:43.604822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:43.635714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b22277d-9e4d-4a9a-9efe-9fb65a3ad9e3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:43.709556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8fae15-e910-4346-b26d-953104a32018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:43.710159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fae15-e910-4346-b26d-953104a32018 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:43.710434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fae15-e910-4346-b26d-953104a32018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:43.710701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8fae15-e910-4346-b26d-953104a32018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:43.712265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8d8fae15-e910-4346-b26d-953104a32018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:43.821364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fae15-e910-4346-b26d-953104a32018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:43.822830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8fae15-e910-4346-b26d-953104a32018 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:43.840091 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 525/1017] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:47.887857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb62a0-d04f-434c-b42e-3d59a42f1625 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:47.888478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb62a0-d04f-434c-b42e-3d59a42f1625 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:47.888478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb62a0-d04f-434c-b42e-3d59a42f1625 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:47.889734 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 493/1018] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:47.895200 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ecda-ae6e-41a1-b9d8-a7de8fe6be8d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:47.895628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ecda-ae6e-41a1-b9d8-a7de8fe6be8d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:47.895736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab26ecda-ae6e-41a1-b9d8-a7de8fe6be8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:47.901338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:47.961806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ab26ecda-ae6e-41a1-b9d8-a7de8fe6be8d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:47.963024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ab26ecda-ae6e-41a1-b9d8-a7de8fe6be8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:48.060272 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 526/1019] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:48.350880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb62a0-d04f-434c-b42e-3d59a42f1625 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:48.352468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-97eb62a0-d04f-434c-b42e-3d59a42f1625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:48.401158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:48.455851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb62a0-d04f-434c-b42e-3d59a42f1625 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:48.530233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e46a1e-6ac9-4b6b-be09-fe82ec12874c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:48.530831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e46a1e-6ac9-4b6b-be09-fe82ec12874c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:48.531104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e46a1e-6ac9-4b6b-be09-fe82ec12874c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:48.531371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e46a1e-6ac9-4b6b-be09-fe82ec12874c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:48.532938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-05e46a1e-6ac9-4b6b-be09-fe82ec12874c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:48.623943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:48.640799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e46a1e-6ac9-4b6b-be09-fe82ec12874c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:48.642227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e46a1e-6ac9-4b6b-be09-fe82ec12874c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:48.660218 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 494/1020] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:51.433768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e537e-050e-4953-920f-d3bca813d52c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:51.434326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e537e-050e-4953-920f-d3bca813d52c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:51.434326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185e537e-050e-4953-920f-d3bca813d52c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:51.435677 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 527/1021] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:51.440891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524782a-cb09-48b0-b13c-8667a0e78f3b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:51.441343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524782a-cb09-48b0-b13c-8667a0e78f3b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:51.441432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4524782a-cb09-48b0-b13c-8667a0e78f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:51.510640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4524782a-cb09-48b0-b13c-8667a0e78f3b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:51.511874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4524782a-cb09-48b0-b13c-8667a0e78f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:51.611228 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 495/1022] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:51.893575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e537e-050e-4953-920f-d3bca813d52c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:51.895112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-185e537e-050e-4953-920f-d3bca813d52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:51.986100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:52.002948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185e537e-050e-4953-920f-d3bca813d52c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:52.081738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df221256-f3d8-4540-9c07-5d2a3fe67597 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:52.082251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221256-f3d8-4540-9c07-5d2a3fe67597 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:52.082523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221256-f3d8-4540-9c07-5d2a3fe67597 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:52.082791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df221256-f3d8-4540-9c07-5d2a3fe67597 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:52.084361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-df221256-f3d8-4540-9c07-5d2a3fe67597 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:52.191355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221256-f3d8-4540-9c07-5d2a3fe67597 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:52.192828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df221256-f3d8-4540-9c07-5d2a3fe67597 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:52.211406 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 528/1023] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:47:56.307323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59dba6-339c-42e9-9fc2-16ec120c9996 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:56.307802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59dba6-339c-42e9-9fc2-16ec120c9996 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:56.307897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd59dba6-339c-42e9-9fc2-16ec120c9996 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:56.309301 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 496/1024] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:47:56.315467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8bcde-4d79-4ecd-9808-47883d51afac None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:56.315866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8bcde-4d79-4ecd-9808-47883d51afac None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:56.316123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8bcde-4d79-4ecd-9808-47883d51afac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:56.331686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:56.385447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6ef8bcde-4d79-4ecd-9808-47883d51afac None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:47:56.386650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6ef8bcde-4d79-4ecd-9808-47883d51afac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:56.480655 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 529/1025] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:47:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:47:56.769174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59dba6-339c-42e9-9fc2-16ec120c9996 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:47:56.770708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cd59dba6-339c-42e9-9fc2-16ec120c9996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:56.827313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:56.873087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd59dba6-339c-42e9-9fc2-16ec120c9996 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:47:56.946167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc40f691-c848-4c58-a25e-c50ac173ec52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:47:56.946786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40f691-c848-4c58-a25e-c50ac173ec52 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:47:56.947044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40f691-c848-4c58-a25e-c50ac173ec52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:47:56.947312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc40f691-c848-4c58-a25e-c50ac173ec52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:47:56.948909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc40f691-c848-4c58-a25e-c50ac173ec52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:47:57.055584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:47:57.059543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40f691-c848-4c58-a25e-c50ac173ec52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:47:57.061377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc40f691-c848-4c58-a25e-c50ac173ec52 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:47:57.079755 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 497/1026] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:47:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:01.941714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31c6b-e91e-40fa-aef2-271e640a114e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:01.942129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31c6b-e91e-40fa-aef2-271e640a114e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:01.942468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df31c6b-e91e-40fa-aef2-271e640a114e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:01.944340 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 530/1027] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:01.949493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d58f8-a294-4102-9cf2-d8c4d964a239 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:01.949934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d58f8-a294-4102-9cf2-d8c4d964a239 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:01.950359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d58f8-a294-4102-9cf2-d8c4d964a239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:02.041158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-162d58f8-a294-4102-9cf2-d8c4d964a239 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:02.042529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-162d58f8-a294-4102-9cf2-d8c4d964a239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:02.221428 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 498/1028] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:48:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:06.458547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33d66-12fc-4c71-8c34-a0aae125221c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:06.459162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33d66-12fc-4c71-8c34-a0aae125221c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:06.459162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33d66-12fc-4c71-8c34-a0aae125221c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:06.460729 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 531/1029] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:06.465849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e6ae-8996-4a93-b0ef-f31cd98da80b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:06.466307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e6ae-8996-4a93-b0ef-f31cd98da80b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:06.466442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208e6ae-8996-4a93-b0ef-f31cd98da80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:06.479056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=59 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:06.524590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:06.548702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d208e6ae-8996-4a93-b0ef-f31cd98da80b None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:06.549905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d208e6ae-8996-4a93-b0ef-f31cd98da80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:06.663117 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 499/1030] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:48:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:07.954413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d8e6-0195-46ab-8283-d0a76f8eb763 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:07.955564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d8e6-0195-46ab-8283-d0a76f8eb763 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:07.955661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55d8e6-0195-46ab-8283-d0a76f8eb763 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:07.957135 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 532/1031] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:48:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:07.962195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d27c3-0283-48f4-8fb0-b2c8f82efbf1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:07.962608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d27c3-0283-48f4-8fb0-b2c8f82efbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:07.962862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962d27c3-0283-48f4-8fb0-b2c8f82efbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:08.037876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-962d27c3-0283-48f4-8fb0-b2c8f82efbf1 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:08.039062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-962d27c3-0283-48f4-8fb0-b2c8f82efbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:08.081101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:08.145984 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 500/1032] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:48:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:11.693027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80569f34-5d2a-42b5-a3c4-ff2d7f3f1345 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:11.694022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80569f34-5d2a-42b5-a3c4-ff2d7f3f1345 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:11.694613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80569f34-5d2a-42b5-a3c4-ff2d7f3f1345 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:11.696654 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 533/1033] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:11.701771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d06bf-359e-4f04-afec-653e7090a7a9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:11.702229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d06bf-359e-4f04-afec-653e7090a7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:11.702330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43d06bf-359e-4f04-afec-653e7090a7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:11.774423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b43d06bf-359e-4f04-afec-653e7090a7a9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:11.775668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b43d06bf-359e-4f04-afec-653e7090a7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:11.878688 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 501/1034] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:48:12.188661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd846a-efd0-4143-9976-c0a15c04fdfe None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:12.189500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd846a-efd0-4143-9976-c0a15c04fdfe None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:12.189784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafd846a-efd0-4143-9976-c0a15c04fdfe None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:12.191283 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 534/1035] 10.222.0.39 () {54 vars in 961 bytes} [Sun Oct 23 00:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:12.200234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36dc23-34f8-4aba-88cb-0249f4e775ff None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:12.201379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36dc23-34f8-4aba-88cb-0249f4e775ff None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:12.202232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36dc23-34f8-4aba-88cb-0249f4e775ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:12.235434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:12.311850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6e36dc23-34f8-4aba-88cb-0249f4e775ff None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:12.313240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6e36dc23-34f8-4aba-88cb-0249f4e775ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:12.459687 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 502/1036] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Oct 23 00:48:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:12.467710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd846a-efd0-4143-9976-c0a15c04fdfe None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:12.469291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fafd846a-efd0-4143-9976-c0a15c04fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:12.627148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafd846a-efd0-4143-9976-c0a15c04fdfe None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:12.704036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:12.745871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3150640-f63e-4d66-be07-ec9617f109e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:12.746469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3150640-f63e-4d66-be07-ec9617f109e9 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:12.746744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3150640-f63e-4d66-be07-ec9617f109e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:12.747011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3150640-f63e-4d66-be07-ec9617f109e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:12.748614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3150640-f63e-4d66-be07-ec9617f109e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:12.846081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3150640-f63e-4d66-be07-ec9617f109e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:12.847662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3150640-f63e-4d66-be07-ec9617f109e9 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:12.865997 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 535/1037] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:15.255584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f196109-b8cc-40b9-b8f5-a1db2507eb4e None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:15.255584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f196109-b8cc-40b9-b8f5-a1db2507eb4e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:15.255584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f196109-b8cc-40b9-b8f5-a1db2507eb4e None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:15.266189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dd719-af61-430c-8567-d8d994db3e45 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:15.266694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dd719-af61-430c-8567-d8d994db3e45 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:15.266694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dd719-af61-430c-8567-d8d994db3e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:15.267712 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 503/1038] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:15 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:15.355971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9a4dd719-af61-430c-8567-d8d994db3e45 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:15.357173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9a4dd719-af61-430c-8567-d8d994db3e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:15.402433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:15.473736 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 536/1039] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:15.767885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f196109-b8cc-40b9-b8f5-a1db2507eb4e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:15.769445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4f196109-b8cc-40b9-b8f5-a1db2507eb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:15.850339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:15.958423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f196109-b8cc-40b9-b8f5-a1db2507eb4e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:16.081173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86d953-e212-4460-8440-28ec4a1a7546 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:16.081785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d953-e212-4460-8440-28ec4a1a7546 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:16.082069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d953-e212-4460-8440-28ec4a1a7546 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:16.082338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d953-e212-4460-8440-28ec4a1a7546 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:16.083916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7c86d953-e212-4460-8440-28ec4a1a7546 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:16.106662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:16.175565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d953-e212-4460-8440-28ec4a1a7546 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:16.177212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d953-e212-4460-8440-28ec4a1a7546 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:16.199783 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 504/1040] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:18.682831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606ae9-da14-401e-968f-a2011cf3ef23 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:18.683363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606ae9-da14-401e-968f-a2011cf3ef23 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:18.683363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1606ae9-da14-401e-968f-a2011cf3ef23 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:18.684818 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 537/1041] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:18.690026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f206f10-cb06-4d80-9bc9-717264da0959 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:18.690456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f206f10-cb06-4d80-9bc9-717264da0959 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:18.690559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f206f10-cb06-4d80-9bc9-717264da0959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:18.760271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0f206f10-cb06-4d80-9bc9-717264da0959 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:18.761492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0f206f10-cb06-4d80-9bc9-717264da0959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:18.868504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:18.876068 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 505/1042] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:19.162003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba71199-b8dc-43e6-912c-c864154a206e None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:19.162356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba71199-b8dc-43e6-912c-c864154a206e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:19.162737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba71199-b8dc-43e6-912c-c864154a206e None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:19.164153 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 538/1043] 10.222.0.39 () {54 vars in 961 bytes} [Sun Oct 23 00:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:19.169473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03418958-2d0d-43a7-8808-8cef4d108a7e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:19.169845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03418958-2d0d-43a7-8808-8cef4d108a7e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:19.170223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03418958-2d0d-43a7-8808-8cef4d108a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:19.247439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-03418958-2d0d-43a7-8808-8cef4d108a7e None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:19.248850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-03418958-2d0d-43a7-8808-8cef4d108a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:19.368787 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 506/1044] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Oct 23 00:48:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:19.377320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba71199-b8dc-43e6-912c-c864154a206e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:19.378833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5ba71199-b8dc-43e6-912c-c864154a206e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:19.532808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba71199-b8dc-43e6-912c-c864154a206e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:19.554364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:19.655730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f90c22-6d03-4276-8338-57f17a596b4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:19.656417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f90c22-6d03-4276-8338-57f17a596b4c service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:19.656697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f90c22-6d03-4276-8338-57f17a596b4c service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:19.656966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f90c22-6d03-4276-8338-57f17a596b4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:19.658562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c7f90c22-6d03-4276-8338-57f17a596b4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:19.751964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f90c22-6d03-4276-8338-57f17a596b4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:19.753600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f90c22-6d03-4276-8338-57f17a596b4c service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:19.772666 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 539/1045] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:19.903584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960004-db13-491e-b2ad-97aa41ccc8d2 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:19.904174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960004-db13-491e-b2ad-97aa41ccc8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:19.904174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a960004-db13-491e-b2ad-97aa41ccc8d2 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:19.905647 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 507/1046] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:19.912194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90ecfc-2ca3-47af-9d19-76f4a07c5188 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:19.912435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90ecfc-2ca3-47af-9d19-76f4a07c5188 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:19.912712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90ecfc-2ca3-47af-9d19-76f4a07c5188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:19.948409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:19.993447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ac90ecfc-2ca3-47af-9d19-76f4a07c5188 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:19.994693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ac90ecfc-2ca3-47af-9d19-76f4a07c5188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:20.116691 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 540/1047] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:20.448174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a7c2e-68a8-4ba3-9280-c2c33f2d7a2f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:20.448601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a7c2e-68a8-4ba3-9280-c2c33f2d7a2f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:20.448963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a7c2e-68a8-4ba3-9280-c2c33f2d7a2f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:20.450410 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 508/1048] 10.222.0.39 () {54 vars in 963 bytes} [Sun Oct 23 00:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:20.455844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02675cf-b0bb-4fa5-8400-a1dcf2cdcfe3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:20.456125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02675cf-b0bb-4fa5-8400-a1dcf2cdcfe3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:20.456440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02675cf-b0bb-4fa5-8400-a1dcf2cdcfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:20.536190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d02675cf-b0bb-4fa5-8400-a1dcf2cdcfe3 None None] MFA Rules not processed for user `9075ccb9e3cd46cc9e77caa4060e3c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:20.537801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d02675cf-b0bb-4fa5-8400-a1dcf2cdcfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:20.588159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:20.621183 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 541/1049] 10.222.0.39 () {58 vars in 1060 bytes} [Sun Oct 23 00:48:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:20.629260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a7c2e-68a8-4ba3-9280-c2c33f2d7a2f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:20.630751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d12a7c2e-68a8-4ba3-9280-c2c33f2d7a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:20.683498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:20.740159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a7c2e-68a8-4ba3-9280-c2c33f2d7a2f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:20.839005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f63f42-a385-49ff-b3b7-031923a55b6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:20.839613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f63f42-a385-49ff-b3b7-031923a55b6b service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:20.839902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f63f42-a385-49ff-b3b7-031923a55b6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:20.840141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f63f42-a385-49ff-b3b7-031923a55b6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:20.841737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e0f63f42-a385-49ff-b3b7-031923a55b6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:20.949712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:20.965926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f63f42-a385-49ff-b3b7-031923a55b6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:20.967386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f63f42-a385-49ff-b3b7-031923a55b6b service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:20.992438 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 509/1050] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:22.036505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872fdc3-da16-412e-9b33-43504f98186f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:22.037040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872fdc3-da16-412e-9b33-43504f98186f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:22.037040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872fdc3-da16-412e-9b33-43504f98186f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:22.038312 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 542/1051] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:22.043967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5118754-50b3-48d0-8844-0b5fcbc6997d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:22.044425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5118754-50b3-48d0-8844-0b5fcbc6997d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:22.044561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5118754-50b3-48d0-8844-0b5fcbc6997d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:22.109172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c5118754-50b3-48d0-8844-0b5fcbc6997d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:22.110367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c5118754-50b3-48d0-8844-0b5fcbc6997d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:22.205344 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 510/1052] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:48:22.494805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872fdc3-da16-412e-9b33-43504f98186f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:22.496923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7872fdc3-da16-412e-9b33-43504f98186f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:22.654977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872fdc3-da16-412e-9b33-43504f98186f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:22.707991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:22.770583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a4f1fe-b1dc-46e7-a776-3e720137248b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:22.771182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f1fe-b1dc-46e7-a776-3e720137248b service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:22.771453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f1fe-b1dc-46e7-a776-3e720137248b service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:22.771711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a4f1fe-b1dc-46e7-a776-3e720137248b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:22.773308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b7a4f1fe-b1dc-46e7-a776-3e720137248b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:22.870398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f1fe-b1dc-46e7-a776-3e720137248b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:22.872037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a4f1fe-b1dc-46e7-a776-3e720137248b service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:22.892227 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 543/1053] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:25.105135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d907-2a90-4101-a715-837fff923bf6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:25.105980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d907-2a90-4101-a715-837fff923bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:25.106259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db7d907-2a90-4101-a715-837fff923bf6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:25.107600 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 511/1054] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:25.113397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855fc8a-b1ec-4626-b2a0-9ceb214eb677 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:25.113849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855fc8a-b1ec-4626-b2a0-9ceb214eb677 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:25.113949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855fc8a-b1ec-4626-b2a0-9ceb214eb677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:25.182860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b855fc8a-b1ec-4626-b2a0-9ceb214eb677 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:25.184055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b855fc8a-b1ec-4626-b2a0-9ceb214eb677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:25.191543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:25.296787 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 544/1055] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:25.584921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d907-2a90-4101-a715-837fff923bf6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:25.586617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6db7d907-2a90-4101-a715-837fff923bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:25.650147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:25.742164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db7d907-2a90-4101-a715-837fff923bf6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:25.860584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1433dfa-e20c-43f1-a643-7b29388528fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:25.861272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1433dfa-e20c-43f1-a643-7b29388528fe service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:25.861623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1433dfa-e20c-43f1-a643-7b29388528fe service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:25.861963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1433dfa-e20c-43f1-a643-7b29388528fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:25.863593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f1433dfa-e20c-43f1-a643-7b29388528fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:25.883316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:25.957473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1433dfa-e20c-43f1-a643-7b29388528fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:25.959147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1433dfa-e20c-43f1-a643-7b29388528fe service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:25.977668 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 512/1056] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:28.196845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67866f-a557-437f-be59-758d80a62aa4 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:28.197404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67866f-a557-437f-be59-758d80a62aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:28.197573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d67866f-a557-437f-be59-758d80a62aa4 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:28.199180 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 545/1057] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:28.204207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c202-ea2f-4503-a58f-298a16bbe6cf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:28.204648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c202-ea2f-4503-a58f-298a16bbe6cf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:28.204833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018c202-ea2f-4503-a58f-298a16bbe6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:28.275681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5018c202-ea2f-4503-a58f-298a16bbe6cf None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:28.277182 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5018c202-ea2f-4503-a58f-298a16bbe6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:28.409039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:28.436133 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 513/1058] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:28.708720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67866f-a557-437f-be59-758d80a62aa4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:28.710157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5d67866f-a557-437f-be59-758d80a62aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:28.851305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:28.861771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d67866f-a557-437f-be59-758d80a62aa4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:28.973452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b91f1f-262b-4daa-8c6c-b428608460a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:28.973991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b91f1f-262b-4daa-8c6c-b428608460a9 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:28.974280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b91f1f-262b-4daa-8c6c-b428608460a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:28.974549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b91f1f-262b-4daa-8c6c-b428608460a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:28.976162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f8b91f1f-262b-4daa-8c6c-b428608460a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:29.065854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b91f1f-262b-4daa-8c6c-b428608460a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:29.067487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b91f1f-262b-4daa-8c6c-b428608460a9 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:29.077335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:29.086640 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 546/1059] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:31.276600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56f63-2290-419e-9081-e4ddbbead844 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:31.277127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56f63-2290-419e-9081-e4ddbbead844 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:31.277127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56f63-2290-419e-9081-e4ddbbead844 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:31.278543 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 514/1060] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:31.284330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4141-cf8c-41cf-a862-72c247ce90f1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:31.284897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4141-cf8c-41cf-a862-72c247ce90f1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:31.284897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c4141-cf8c-41cf-a862-72c247ce90f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:31.353833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e87c4141-cf8c-41cf-a862-72c247ce90f1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:31.355050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e87c4141-cf8c-41cf-a862-72c247ce90f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:31.456290 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 547/1061] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:31.832865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a56f63-2290-419e-9081-e4ddbbead844 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:31.834940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-12a56f63-2290-419e-9081-e4ddbbead844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:31.989749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a56f63-2290-419e-9081-e4ddbbead844 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:32.062469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:32.110029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af94775-f10d-4286-a734-26a5161a191a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:32.111153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af94775-f10d-4286-a734-26a5161a191a service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:32.111430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af94775-f10d-4286-a734-26a5161a191a service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:32.111701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af94775-f10d-4286-a734-26a5161a191a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:32.113320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2af94775-f10d-4286-a734-26a5161a191a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:32.203591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af94775-f10d-4286-a734-26a5161a191a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:32.205843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af94775-f10d-4286-a734-26a5161a191a service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:32.226957 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 515/1062] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:34.410219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9f52a-4130-42bf-93cc-bda76090e157 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:34.410946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9f52a-4130-42bf-93cc-bda76090e157 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:34.410946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9f52a-4130-42bf-93cc-bda76090e157 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:34.412140 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 548/1063] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:34.417749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32597be6-d97d-4132-b6b2-c76e423d9c19 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:34.418161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32597be6-d97d-4132-b6b2-c76e423d9c19 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:34.418288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32597be6-d97d-4132-b6b2-c76e423d9c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:34.484869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-32597be6-d97d-4132-b6b2-c76e423d9c19 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:34.486624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-32597be6-d97d-4132-b6b2-c76e423d9c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:34.520459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:34.588718 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 516/1064] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:34.880385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9f52a-4130-42bf-93cc-bda76090e157 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:34.881844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-58d9f52a-4130-42bf-93cc-bda76090e157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:34.959285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:35.034675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9f52a-4130-42bf-93cc-bda76090e157 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:35.161756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a7c02-0517-4095-91cb-36eadda444cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:35.162472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a7c02-0517-4095-91cb-36eadda444cd service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:35.162854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a7c02-0517-4095-91cb-36eadda444cd service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:35.163190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a7c02-0517-4095-91cb-36eadda444cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:35.165308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-691a7c02-0517-4095-91cb-36eadda444cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:35.206528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:35.258328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a7c02-0517-4095-91cb-36eadda444cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:35.260342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a7c02-0517-4095-91cb-36eadda444cd service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:35.279338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 549/1065] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:37.291004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ade11-9532-40b8-b61e-3a020353e769 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:37.291562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ade11-9532-40b8-b61e-3a020353e769 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:37.291562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ade11-9532-40b8-b61e-3a020353e769 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:37.292952 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 517/1066] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:37.298942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16cd8-115d-4658-8815-f82b65c57acc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:37.299401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16cd8-115d-4658-8815-f82b65c57acc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:37.299505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a16cd8-115d-4658-8815-f82b65c57acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:37.367924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e6a16cd8-115d-4658-8815-f82b65c57acc None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:37.369192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e6a16cd8-115d-4658-8815-f82b65c57acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:37.466390 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 550/1067] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:37.473367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d54924-1196-4eca-8b19-9a7256d990b1 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:37.473716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d54924-1196-4eca-8b19-9a7256d990b1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:37.474090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d54924-1196-4eca-8b19-9a7256d990b1 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:37.475508 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 518/1068] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:37.480783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218d1c-6097-417a-9a60-e5c6edf9bdd7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:37.481061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218d1c-6097-417a-9a60-e5c6edf9bdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:37.481327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94218d1c-6097-417a-9a60-e5c6edf9bdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:37.496754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:37.549165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-94218d1c-6097-417a-9a60-e5c6edf9bdd7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:37.550374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-94218d1c-6097-417a-9a60-e5c6edf9bdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:37.644519 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 551/1069] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:37.770472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d048e-c73a-4597-8639-efa76bd14292 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:37.770827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d048e-c73a-4597-8639-efa76bd14292 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:37.771386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d048e-c73a-4597-8639-efa76bd14292 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:37.772927 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 519/1070] 10.222.0.39 () {54 vars in 962 bytes} [Sun Oct 23 00:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:37.778258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2f12d-22b8-463a-9a87-7e27cb2b0ac6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:37.778545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2f12d-22b8-463a-9a87-7e27cb2b0ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:37.778818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2f12d-22b8-463a-9a87-7e27cb2b0ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:37.859790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f9e2f12d-22b8-463a-9a87-7e27cb2b0ac6 None None] MFA Rules not processed for user `b5429182e2fa4e2aa485fc084e32aafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:37.861057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f9e2f12d-22b8-463a-9a87-7e27cb2b0ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:37.896067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:37.934946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d048e-c73a-4597-8639-efa76bd14292 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:37.936569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f68d048e-c73a-4597-8639-efa76bd14292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:37.940575 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 552/1071] 10.222.0.39 () {58 vars in 1059 bytes} [Sun Oct 23 00:48:37 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:37.948123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2f12d-22b8-463a-9a87-7e27cb2b0ac6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:37.956125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2f12d-22b8-463a-9a87-7e27cb2b0ac6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:38.029389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5915862-4b17-4d39-a7e9-f5ef20c3a3f6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:38.029990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915862-4b17-4d39-a7e9-f5ef20c3a3f6 service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:38.030260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915862-4b17-4d39-a7e9-f5ef20c3a3f6 service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:38.030523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5915862-4b17-4d39-a7e9-f5ef20c3a3f6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:38.032242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d5915862-4b17-4d39-a7e9-f5ef20c3a3f6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:38.081473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d048e-c73a-4597-8639-efa76bd14292 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:38.097380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:38.138145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:38.141919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5915862-4b17-4d39-a7e9-f5ef20c3a3f6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:38.143650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5915862-4b17-4d39-a7e9-f5ef20c3a3f6 service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:38.162140 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 553/1072] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:48:38.196908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf30b6-8888-4eaa-8a56-6a96fce66892 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:38.197508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf30b6-8888-4eaa-8a56-6a96fce66892 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:38.197781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf30b6-8888-4eaa-8a56-6a96fce66892 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:38.198045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf30b6-8888-4eaa-8a56-6a96fce66892 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:38.199594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3bdf30b6-8888-4eaa-8a56-6a96fce66892 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:38.288116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf30b6-8888-4eaa-8a56-6a96fce66892 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:38.289739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf30b6-8888-4eaa-8a56-6a96fce66892 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:38.307963 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 520/1073] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:48:38.909087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40c67d-ffa1-49af-8c80-9808d0f52363 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:38.909559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40c67d-ffa1-49af-8c80-9808d0f52363 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:38.909804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40c67d-ffa1-49af-8c80-9808d0f52363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:38.975164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1d40c67d-ffa1-49af-8c80-9808d0f52363 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:38.976275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1d40c67d-ffa1-49af-8c80-9808d0f52363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:39.062616 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 554/1074] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:48:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:39.106148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec24aa2-4519-4208-a97f-92647553d290 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:39.106431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec24aa2-4519-4208-a97f-92647553d290 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:39.106701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec24aa2-4519-4208-a97f-92647553d290 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:39.108041 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 521/1075] 10.222.0.39 () {54 vars in 943 bytes} [Sun Oct 23 00:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:39.114784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668e085-1b6a-4601-b9d5-45dbbd2cb0e7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:39.115096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668e085-1b6a-4601-b9d5-45dbbd2cb0e7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:39.115370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2668e085-1b6a-4601-b9d5-45dbbd2cb0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:39.190765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2668e085-1b6a-4601-b9d5-45dbbd2cb0e7 None None] MFA Rules not processed for user `0800a1f4140a4d098eadb94eeba922be`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:39.191936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2668e085-1b6a-4601-b9d5-45dbbd2cb0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:39.208312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:39.298411 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 555/1076] 10.222.0.39 () {58 vars in 1040 bytes} [Sun Oct 23 00:48:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:39.307375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec24aa2-4519-4208-a97f-92647553d290 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:39.308882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4ec24aa2-4519-4208-a97f-92647553d290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:39.331992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:39.444533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec24aa2-4519-4208-a97f-92647553d290 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:39.550167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b5b1a-617c-4a75-8eed-3feeab0021bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:39.550775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b5b1a-617c-4a75-8eed-3feeab0021bd service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:39.551055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b5b1a-617c-4a75-8eed-3feeab0021bd service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:39.551324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b5b1a-617c-4a75-8eed-3feeab0021bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:39.552964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5c2b5b1a-617c-4a75-8eed-3feeab0021bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:39.556337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:39.635931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b5b1a-617c-4a75-8eed-3feeab0021bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:39.637536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b5b1a-617c-4a75-8eed-3feeab0021bd service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:39.646035 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 522/1077] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 00:48:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:48:40.530637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd2c7-0d3c-4834-a5f8-768901386f85 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:40.531210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd2c7-0d3c-4834-a5f8-768901386f85 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:40.531210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4dd2c7-0d3c-4834-a5f8-768901386f85 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:40.538913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27bf6c-1e48-4bf7-9d06-6bfa33b412c6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:40.539431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27bf6c-1e48-4bf7-9d06-6bfa33b412c6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:40.539523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27bf6c-1e48-4bf7-9d06-6bfa33b412c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:40.542172 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 556/1078] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:40.609977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5f27bf6c-1e48-4bf7-9d06-6bfa33b412c6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:40.611564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f27bf6c-1e48-4bf7-9d06-6bfa33b412c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:40.710521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:40.724346 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 523/1079] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:41.002805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dd2c7-0d3c-4834-a5f8-768901386f85 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:41.004420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fc4dd2c7-0d3c-4834-a5f8-768901386f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:41.158068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:41.170040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4dd2c7-0d3c-4834-a5f8-768901386f85 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:41.290816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443b5db5-93c5-47dc-80b2-da3f66e5764b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:41.291798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5db5-93c5-47dc-80b2-da3f66e5764b service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:41.292072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5db5-93c5-47dc-80b2-da3f66e5764b service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:41.292364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443b5db5-93c5-47dc-80b2-da3f66e5764b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:41.293950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-443b5db5-93c5-47dc-80b2-da3f66e5764b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:41.392916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b5db5-93c5-47dc-80b2-da3f66e5764b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:41.394648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443b5db5-93c5-47dc-80b2-da3f66e5764b service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:41.403886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:41.414883 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 557/1080] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:43.640906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1cfa-2059-45be-85c1-fedf0769b8f5 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:43.641489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1cfa-2059-45be-85c1-fedf0769b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:43.641489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1cfa-2059-45be-85c1-fedf0769b8f5 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:43.647011 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 524/1081] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:43.648909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f2859-3bb5-4e84-ae0e-8e89ced9eb83 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:43.649953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f2859-3bb5-4e84-ae0e-8e89ced9eb83 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:43.650432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5f2859-3bb5-4e84-ae0e-8e89ced9eb83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:43.730482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-eb5f2859-3bb5-4e84-ae0e-8e89ced9eb83 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:43.731868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eb5f2859-3bb5-4e84-ae0e-8e89ced9eb83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:43.852257 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 558/1082] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:44.196115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1cfa-2059-45be-85c1-fedf0769b8f5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:44.198231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bd4e1cfa-2059-45be-85c1-fedf0769b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:44.372233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1cfa-2059-45be-85c1-fedf0769b8f5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:44.454657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:44.500858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c597d-bcf2-4730-8805-68f97c40f5ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:44.501379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c597d-bcf2-4730-8805-68f97c40f5ff service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:44.501657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c597d-bcf2-4730-8805-68f97c40f5ff service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:44.501997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c597d-bcf2-4730-8805-68f97c40f5ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:44.503506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fc5c597d-bcf2-4730-8805-68f97c40f5ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:44.608860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c597d-bcf2-4730-8805-68f97c40f5ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:44.611126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c597d-bcf2-4730-8805-68f97c40f5ff service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:44.632931 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 525/1083] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:46.840411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9a36-0e4d-4711-a8e6-b272c2365e56 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:46.840892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9a36-0e4d-4711-a8e6-b272c2365e56 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:46.840995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df9a36-0e4d-4711-a8e6-b272c2365e56 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:46.846051 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 559/1084] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:48:46.848475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd9e51-d9b4-49ce-84aa-e7e9250d89a7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:46.848913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd9e51-d9b4-49ce-84aa-e7e9250d89a7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:46.849026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdd9e51-d9b4-49ce-84aa-e7e9250d89a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:46.922558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2bdd9e51-d9b4-49ce-84aa-e7e9250d89a7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:48:46.923771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2bdd9e51-d9b4-49ce-84aa-e7e9250d89a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:46.960013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:47.033338 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 526/1085] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:48:47.325427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df9a36-0e4d-4711-a8e6-b272c2365e56 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:48:47.326940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f9df9a36-0e4d-4711-a8e6-b272c2365e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:47.410418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:47.491139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df9a36-0e4d-4711-a8e6-b272c2365e56 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:48:47.630524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28a950a-b03c-4cfa-9795-5c604c7b8126 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:48:47.631057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a950a-b03c-4cfa-9795-5c604c7b8126 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:48:47.631338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a950a-b03c-4cfa-9795-5c604c7b8126 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:48:47.631614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28a950a-b03c-4cfa-9795-5c604c7b8126 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:48:47.633254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a28a950a-b03c-4cfa-9795-5c604c7b8126 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:48:47.671716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:47.733724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a950a-b03c-4cfa-9795-5c604c7b8126 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:48:47.736113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28a950a-b03c-4cfa-9795-5c604c7b8126 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:48:47.756779 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 560/1086] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:48:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:48:57.031874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=28 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:48:57.753814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=30 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:02.311226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b4bb-e7c7-47d7-95d0-a1ce83d21561 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:02.311775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b4bb-e7c7-47d7-95d0-a1ce83d21561 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:02.311775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196b4bb-e7c7-47d7-95d0-a1ce83d21561 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:02.313140 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 527/1087] 10.222.0.39 () {54 vars in 918 bytes} [Sun Oct 23 00:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:02.318393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a000-618d-4698-924b-e0a20d7e3614 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:02.318833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a000-618d-4698-924b-e0a20d7e3614 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:02.319084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d06a000-618d-4698-924b-e0a20d7e3614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:02.394589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0d06a000-618d-4698-924b-e0a20d7e3614 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:02.395792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0d06a000-618d-4698-924b-e0a20d7e3614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:02.500516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 561/1088] 10.222.0.39 () {58 vars in 1015 bytes} [Sun Oct 23 00:49:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:02.514853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c91c4-a26a-4ade-8d3b-ec9a9cdd3a81 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:02.515135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c91c4-a26a-4ade-8d3b-ec9a9cdd3a81 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:02.515407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7c91c4-a26a-4ade-8d3b-ec9a9cdd3a81 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:02.517278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 528/1089] 10.222.0.39 () {54 vars in 918 bytes} [Sun Oct 23 00:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:02.521724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f6aaa-8380-4ed0-9371-4e8744930e99 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:02.522005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f6aaa-8380-4ed0-9371-4e8744930e99 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:02.522272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f6aaa-8380-4ed0-9371-4e8744930e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:02.595448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:02.598144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9e5f6aaa-8380-4ed0-9371-4e8744930e99 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:02.599967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9e5f6aaa-8380-4ed0-9371-4e8744930e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:02.713356 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 562/1090] 10.222.0.39 () {58 vars in 1015 bytes} [Sun Oct 23 00:49:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:49:05.256656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b573d-1006-4fef-82c2-d50c02b9075a None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:05.257220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b573d-1006-4fef-82c2-d50c02b9075a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:05.257220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b573d-1006-4fef-82c2-d50c02b9075a None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:05.258534 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 529/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:05.263925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8efa-f09d-4227-ad7a-340d5ea90cef None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:05.264357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8efa-f09d-4227-ad7a-340d5ea90cef None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:05.264486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a8efa-f09d-4227-ad7a-340d5ea90cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:05.328647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b69a8efa-f09d-4227-ad7a-340d5ea90cef None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:05.329748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b69a8efa-f09d-4227-ad7a-340d5ea90cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:05.382128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:05.422588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 563/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:05.709251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b573d-1006-4fef-82c2-d50c02b9075a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:05.710806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5d3b573d-1006-4fef-82c2-d50c02b9075a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:05.853631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b573d-1006-4fef-82c2-d50c02b9075a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:05.924532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:05.964644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a72d7c-d4c7-4f4a-8f7f-ca4fe3d0473c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:05.965238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72d7c-d4c7-4f4a-8f7f-ca4fe3d0473c service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:05.965517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72d7c-d4c7-4f4a-8f7f-ca4fe3d0473c service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:05.965780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a72d7c-d4c7-4f4a-8f7f-ca4fe3d0473c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:05.967325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c3a72d7c-d4c7-4f4a-8f7f-ca4fe3d0473c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:06.052881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a72d7c-d4c7-4f4a-8f7f-ca4fe3d0473c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:06.054454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a72d7c-d4c7-4f4a-8f7f-ca4fe3d0473c service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:06.072162 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 530/1093] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:07.203813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12012bdd-a61e-4f9d-83fb-05ec7595868f None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:07.204382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12012bdd-a61e-4f9d-83fb-05ec7595868f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:07.204382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12012bdd-a61e-4f9d-83fb-05ec7595868f None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:07.205652 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 564/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:07.210655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb6320-2514-4109-9db4-74eda15c2da7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:07.211096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb6320-2514-4109-9db4-74eda15c2da7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:07.211200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb6320-2514-4109-9db4-74eda15c2da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:07.286705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-97eb6320-2514-4109-9db4-74eda15c2da7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:07.287884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-97eb6320-2514-4109-9db4-74eda15c2da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:07.314322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:07.383278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 531/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:07.535933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12012bdd-a61e-4f9d-83fb-05ec7595868f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:07.537456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-12012bdd-a61e-4f9d-83fb-05ec7595868f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:07.682169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12012bdd-a61e-4f9d-83fb-05ec7595868f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:07.723435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:07.796236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7917c33f-6ac3-49a4-abc9-38cc78d309d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:07.796830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c33f-6ac3-49a4-abc9-38cc78d309d0 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:07.797110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c33f-6ac3-49a4-abc9-38cc78d309d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:07.797380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7917c33f-6ac3-49a4-abc9-38cc78d309d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:07.798965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7917c33f-6ac3-49a4-abc9-38cc78d309d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:07.887553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917c33f-6ac3-49a4-abc9-38cc78d309d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:07.889177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7917c33f-6ac3-49a4-abc9-38cc78d309d0 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:07.907172 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 565/1096] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:09.021834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b3fb9-5d1a-44d4-9e84-5fae724edb88 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:09.022373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b3fb9-5d1a-44d4-9e84-5fae724edb88 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:09.022461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b3fb9-5d1a-44d4-9e84-5fae724edb88 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:09.023792 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 532/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:09.028565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d8084-3c30-45c7-b236-ed53afe2c971 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:09.028967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d8084-3c30-45c7-b236-ed53afe2c971 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:09.029112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4d8084-3c30-45c7-b236-ed53afe2c971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:09.082870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:09.104631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-be4d8084-3c30-45c7-b236-ed53afe2c971 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:09.105823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be4d8084-3c30-45c7-b236-ed53afe2c971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:09.198590 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 566/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:09.352332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b3fb9-5d1a-44d4-9e84-5fae724edb88 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:09.353755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-538b3fb9-5d1a-44d4-9e84-5fae724edb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:09.495260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b3fb9-5d1a-44d4-9e84-5fae724edb88 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:09.507710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:09.605308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2bb8d-b287-44cb-8c0d-f9e2114625e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:09.605825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2bb8d-b287-44cb-8c0d-f9e2114625e8 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:09.606099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2bb8d-b287-44cb-8c0d-f9e2114625e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:09.606363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2bb8d-b287-44cb-8c0d-f9e2114625e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:09.607905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1af2bb8d-b287-44cb-8c0d-f9e2114625e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:09.695758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2bb8d-b287-44cb-8c0d-f9e2114625e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:09.697430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2bb8d-b287-44cb-8c0d-f9e2114625e8 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:09.715262 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 533/1099] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:10.811359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546ea6-f45c-473e-9841-f2c78b3231e6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:10.811920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546ea6-f45c-473e-9841-f2c78b3231e6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:10.811920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546ea6-f45c-473e-9841-f2c78b3231e6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:10.813722 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 567/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:10.818921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe822c-7096-4f54-95d1-227412d64c1a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:10.819349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe822c-7096-4f54-95d1-227412d64c1a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:10.819445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe822c-7096-4f54-95d1-227412d64c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:10.839499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:10.896754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-55fe822c-7096-4f54-95d1-227412d64c1a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:10.897925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-55fe822c-7096-4f54-95d1-227412d64c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:10.990821 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 534/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:11.142029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d546ea6-f45c-473e-9841-f2c78b3231e6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:11.143528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0d546ea6-f45c-473e-9841-f2c78b3231e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:11.250869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:11.285633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d546ea6-f45c-473e-9841-f2c78b3231e6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:11.395414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2e753-7e0f-476e-95b3-ab73c7dda22f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:11.395936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e753-7e0f-476e-95b3-ab73c7dda22f service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:11.396208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e753-7e0f-476e-95b3-ab73c7dda22f service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:11.396497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e753-7e0f-476e-95b3-ab73c7dda22f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:11.398046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e9c2e753-7e0f-476e-95b3-ab73c7dda22f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:11.471690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:11.486023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2e753-7e0f-476e-95b3-ab73c7dda22f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:11.487598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2e753-7e0f-476e-95b3-ab73c7dda22f service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:11.505739 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 568/1102] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:12.589853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86cb06-5456-41a8-826e-6e35cf539290 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:12.590394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86cb06-5456-41a8-826e-6e35cf539290 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:12.590394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86cb06-5456-41a8-826e-6e35cf539290 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:12.591729 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 535/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:12.596495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf270f0-594b-4a63-90c5-5a554140ce31 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:12.596893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf270f0-594b-4a63-90c5-5a554140ce31 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:12.597034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf270f0-594b-4a63-90c5-5a554140ce31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:12.670706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-caf270f0-594b-4a63-90c5-5a554140ce31 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:12.672412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-caf270f0-594b-4a63-90c5-5a554140ce31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:12.771016 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 569/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:12.919161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86cb06-5456-41a8-826e-6e35cf539290 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:12.920798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0a86cb06-5456-41a8-826e-6e35cf539290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:13.004284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:13.064508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86cb06-5456-41a8-826e-6e35cf539290 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:13.173881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488da440-941a-446c-9047-e34b81c996fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:13.174402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da440-941a-446c-9047-e34b81c996fc service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:13.174674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da440-941a-446c-9047-e34b81c996fc service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:13.174939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488da440-941a-446c-9047-e34b81c996fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:13.176500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-488da440-941a-446c-9047-e34b81c996fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:13.223119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:13.264229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488da440-941a-446c-9047-e34b81c996fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:13.265872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488da440-941a-446c-9047-e34b81c996fc service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:13.283580 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 536/1105] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:14.317778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90c51-7f50-4545-8851-8aeaf42e68b1 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:14.318317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90c51-7f50-4545-8851-8aeaf42e68b1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:14.318317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef90c51-7f50-4545-8851-8aeaf42e68b1 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:14.319691 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 570/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:14.324665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93328e0c-d052-4186-bf48-bcd6afdb2bbd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:14.325072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93328e0c-d052-4186-bf48-bcd6afdb2bbd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:14.325204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93328e0c-d052-4186-bf48-bcd6afdb2bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:14.398653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-93328e0c-d052-4186-bf48-bcd6afdb2bbd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:14.399844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-93328e0c-d052-4186-bf48-bcd6afdb2bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:14.493223 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 537/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:14.646394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90c51-7f50-4545-8851-8aeaf42e68b1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:14.647828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fef90c51-7f50-4545-8851-8aeaf42e68b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:14.696121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:14.788520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef90c51-7f50-4545-8851-8aeaf42e68b1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:14.900396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84a61d1-00bb-4c38-937b-2b73964d38cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:14.900935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a61d1-00bb-4c38-937b-2b73964d38cb service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:14.901209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a61d1-00bb-4c38-937b-2b73964d38cb service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:14.901476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84a61d1-00bb-4c38-937b-2b73964d38cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:14.903071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a84a61d1-00bb-4c38-937b-2b73964d38cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:14.910442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:14.995120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a61d1-00bb-4c38-937b-2b73964d38cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:14.996739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84a61d1-00bb-4c38-937b-2b73964d38cb service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:15.014329 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 571/1108] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:16.049637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1143cd0-3cea-4f2f-b3f9-bdb977868320 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:16.050203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1143cd0-3cea-4f2f-b3f9-bdb977868320 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:16.050203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1143cd0-3cea-4f2f-b3f9-bdb977868320 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:16.051531 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 538/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:16.056269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9b1c-7993-40cd-8f5c-29e468e82ca3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:16.056742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9b1c-7993-40cd-8f5c-29e468e82ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:16.056847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b9b1c-7993-40cd-8f5c-29e468e82ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:16.131762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-076b9b1c-7993-40cd-8f5c-29e468e82ca3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:16.132896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-076b9b1c-7993-40cd-8f5c-29e468e82ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:16.197953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:16.227457 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 572/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:16.378311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1143cd0-3cea-4f2f-b3f9-bdb977868320 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:16.379793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d1143cd0-3cea-4f2f-b3f9-bdb977868320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:16.397034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:16.521936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1143cd0-3cea-4f2f-b3f9-bdb977868320 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:16.607207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:16.633388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75eefc-75b0-4a89-ac8e-801abfbb2b08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:16.633992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75eefc-75b0-4a89-ac8e-801abfbb2b08 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:16.634273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75eefc-75b0-4a89-ac8e-801abfbb2b08 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:16.634545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75eefc-75b0-4a89-ac8e-801abfbb2b08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:16.636387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ba75eefc-75b0-4a89-ac8e-801abfbb2b08 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:16.723978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75eefc-75b0-4a89-ac8e-801abfbb2b08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:16.725524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75eefc-75b0-4a89-ac8e-801abfbb2b08 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:16.743460 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 539/1111] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:17.777947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55bf75-3d83-4100-88ae-86e52a3bfc4c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:17.778397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55bf75-3d83-4100-88ae-86e52a3bfc4c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:17.778503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55bf75-3d83-4100-88ae-86e52a3bfc4c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:17.779836 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 573/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:17.784764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926165-6167-4207-b232-6acd36f1c30a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:17.785173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926165-6167-4207-b232-6acd36f1c30a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:17.785302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b926165-6167-4207-b232-6acd36f1c30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:17.860770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2b926165-6167-4207-b232-6acd36f1c30a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:17.861990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2b926165-6167-4207-b232-6acd36f1c30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:17.903500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:17.957135 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 540/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:18.104245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55bf75-3d83-4100-88ae-86e52a3bfc4c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:18.105745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ec55bf75-3d83-4100-88ae-86e52a3bfc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:18.246220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55bf75-3d83-4100-88ae-86e52a3bfc4c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:18.298436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:18.355387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a59546e-455c-4a52-a7f7-883867e25d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:18.355917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59546e-455c-4a52-a7f7-883867e25d35 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:18.356194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59546e-455c-4a52-a7f7-883867e25d35 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:18.356493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a59546e-455c-4a52-a7f7-883867e25d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:18.358067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7a59546e-455c-4a52-a7f7-883867e25d35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:18.445489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59546e-455c-4a52-a7f7-883867e25d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:18.447064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a59546e-455c-4a52-a7f7-883867e25d35 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:18.464450 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 574/1114] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:19.503688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363d0e2-5781-454b-b530-92b14d654613 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:19.504218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363d0e2-5781-454b-b530-92b14d654613 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:19.504218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2363d0e2-5781-454b-b530-92b14d654613 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:19.505756 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 541/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:49:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:19.510915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aacd9-e9ef-4a92-9aa4-9de0d6b9c063 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:19.511346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aacd9-e9ef-4a92-9aa4-9de0d6b9c063 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:19.511487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297aacd9-e9ef-4a92-9aa4-9de0d6b9c063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:19.585250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-297aacd9-e9ef-4a92-9aa4-9de0d6b9c063 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:19.586437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-297aacd9-e9ef-4a92-9aa4-9de0d6b9c063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:19.588427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:19.682484 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 575/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:19.832695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363d0e2-5781-454b-b530-92b14d654613 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:19.834199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2363d0e2-5781-454b-b530-92b14d654613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:19.978471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2363d0e2-5781-454b-b530-92b14d654613 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:20.004973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:20.087985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f23792-2c42-42c2-a526-31dc994146a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:20.088541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23792-2c42-42c2-a526-31dc994146a9 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:20.088816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23792-2c42-42c2-a526-31dc994146a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:20.089084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f23792-2c42-42c2-a526-31dc994146a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:20.090627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-16f23792-2c42-42c2-a526-31dc994146a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:20.178077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f23792-2c42-42c2-a526-31dc994146a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:20.179631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f23792-2c42-42c2-a526-31dc994146a9 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:20.196849 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 542/1117] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:22.269134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d402c1-3f34-43fa-95f7-13b6dc629f3a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:22.269677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d402c1-3f34-43fa-95f7-13b6dc629f3a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:22.269677 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d402c1-3f34-43fa-95f7-13b6dc629f3a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:22.271007 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 576/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:22.276412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c46a-6e78-454c-a7a3-77b9529af4cd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:22.276857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c46a-6e78-454c-a7a3-77b9529af4cd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:22.276976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c741c46a-6e78-454c-a7a3-77b9529af4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:22.316414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:22.341607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c741c46a-6e78-454c-a7a3-77b9529af4cd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:22.342792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c741c46a-6e78-454c-a7a3-77b9529af4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:22.436723 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 543/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:22.723083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d402c1-3f34-43fa-95f7-13b6dc629f3a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:22.724674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b3d402c1-3f34-43fa-95f7-13b6dc629f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:22.826091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d402c1-3f34-43fa-95f7-13b6dc629f3a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:22.857700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:22.898706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7d35d-491e-4857-a570-b1123b0dc7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:22.899426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7d35d-491e-4857-a570-b1123b0dc7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:22.899706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7d35d-491e-4857-a570-b1123b0dc7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:22.899974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7d35d-491e-4857-a570-b1123b0dc7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:22.901564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1ea7d35d-491e-4857-a570-b1123b0dc7f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:23.007116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7d35d-491e-4857-a570-b1123b0dc7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:23.008494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7d35d-491e-4857-a570-b1123b0dc7f1 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:23.026131 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 577/1120] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:25.315820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb03e8-fa4a-4aff-8c00-2825fcb25295 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:25.316352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb03e8-fa4a-4aff-8c00-2825fcb25295 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:25.316480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb03e8-fa4a-4aff-8c00-2825fcb25295 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:25.317755 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 544/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:25.322900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7800a4-9ad9-498b-b260-298ff70df0d4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:25.323325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7800a4-9ad9-498b-b260-298ff70df0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:25.323446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7800a4-9ad9-498b-b260-298ff70df0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:25.388386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8e7800a4-9ad9-498b-b260-298ff70df0d4 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:25.389518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e7800a4-9ad9-498b-b260-298ff70df0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:25.402425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:25.485136 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 578/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:25.772614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb03e8-fa4a-4aff-8c00-2825fcb25295 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:25.774120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6acb03e8-fa4a-4aff-8c00-2825fcb25295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:25.878142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb03e8-fa4a-4aff-8c00-2825fcb25295 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:25.890786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:25.950371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35beb7-87f6-424e-957d-dbfe6e21a238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:25.950966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35beb7-87f6-424e-957d-dbfe6e21a238 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:25.951240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35beb7-87f6-424e-957d-dbfe6e21a238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:25.951508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35beb7-87f6-424e-957d-dbfe6e21a238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:25.953127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4f35beb7-87f6-424e-957d-dbfe6e21a238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:26.059514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35beb7-87f6-424e-957d-dbfe6e21a238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:26.060960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35beb7-87f6-424e-957d-dbfe6e21a238 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:26.078862 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 545/1123] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:28.519615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36048571-71c9-4af2-b57d-e36102104db0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:28.520157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36048571-71c9-4af2-b57d-e36102104db0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:28.520157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36048571-71c9-4af2-b57d-e36102104db0 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:28.521576 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 579/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:28.526906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f427a-d6b1-48e3-a3a0-cf65743e94b5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:28.527327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f427a-d6b1-48e3-a3a0-cf65743e94b5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:28.527424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f427a-d6b1-48e3-a3a0-cf65743e94b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:28.578133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:28.595458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b44f427a-d6b1-48e3-a3a0-cf65743e94b5 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:28.596666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b44f427a-d6b1-48e3-a3a0-cf65743e94b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:28.692571 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 546/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:28.974788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36048571-71c9-4af2-b57d-e36102104db0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:28.976374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-36048571-71c9-4af2-b57d-e36102104db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:29.077368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36048571-71c9-4af2-b57d-e36102104db0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:29.117319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:29.150907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eb109a-3949-47ca-8f6f-2e2f3b99ba3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:29.151510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb109a-3949-47ca-8f6f-2e2f3b99ba3f service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:29.151788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb109a-3949-47ca-8f6f-2e2f3b99ba3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:29.152056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eb109a-3949-47ca-8f6f-2e2f3b99ba3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:29.153673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d7eb109a-3949-47ca-8f6f-2e2f3b99ba3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:29.258183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb109a-3949-47ca-8f6f-2e2f3b99ba3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:29.259605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eb109a-3949-47ca-8f6f-2e2f3b99ba3f service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:29.277708 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 580/1126] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:31.544608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd9ca-024a-49cf-a44b-cca368671726 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:31.545137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd9ca-024a-49cf-a44b-cca368671726 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:31.545137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1dd9ca-024a-49cf-a44b-cca368671726 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:31.546397 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 547/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:31.551615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557e801-d441-49ab-bfbe-c45af2a67796 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:31.552019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557e801-d441-49ab-bfbe-c45af2a67796 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:31.552166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557e801-d441-49ab-bfbe-c45af2a67796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:31.617163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2557e801-d441-49ab-bfbe-c45af2a67796 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:31.618356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2557e801-d441-49ab-bfbe-c45af2a67796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:31.638335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:31.712871 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 581/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:31.999978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd9ca-024a-49cf-a44b-cca368671726 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:32.001582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8d1dd9ca-024a-49cf-a44b-cca368671726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:32.102410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1dd9ca-024a-49cf-a44b-cca368671726 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:32.122079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:32.175366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1556c76-2847-45ab-8276-4fd2db0e0b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:32.175983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1556c76-2847-45ab-8276-4fd2db0e0b53 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:32.176254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1556c76-2847-45ab-8276-4fd2db0e0b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:32.176549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1556c76-2847-45ab-8276-4fd2db0e0b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:32.178096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b1556c76-2847-45ab-8276-4fd2db0e0b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:32.287727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1556c76-2847-45ab-8276-4fd2db0e0b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:32.289185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1556c76-2847-45ab-8276-4fd2db0e0b53 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:32.306998 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 548/1129] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:34.504289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d864e9-2146-4778-afcb-b612383a2f3d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:34.504978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d864e9-2146-4778-afcb-b612383a2f3d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:34.504978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d864e9-2146-4778-afcb-b612383a2f3d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:34.506201 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 582/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:34.511916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79367a-52e9-4a75-81aa-d5b4a9c0d016 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:34.512375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79367a-52e9-4a75-81aa-d5b4a9c0d016 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:34.512484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79367a-52e9-4a75-81aa-d5b4a9c0d016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:34.573819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:34.576595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3d79367a-52e9-4a75-81aa-d5b4a9c0d016 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:34.578361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3d79367a-52e9-4a75-81aa-d5b4a9c0d016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:34.679824 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 549/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:34.966382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d864e9-2146-4778-afcb-b612383a2f3d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:34.968485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3d864e9-2146-4778-afcb-b612383a2f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:35.069718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d864e9-2146-4778-afcb-b612383a2f3d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:35.119183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:35.143517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99f7c2b-f8db-4d80-b950-69697596a75d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:35.144117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7c2b-f8db-4d80-b950-69697596a75d service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:35.144419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7c2b-f8db-4d80-b950-69697596a75d service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:35.144691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99f7c2b-f8db-4d80-b950-69697596a75d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:35.146250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d99f7c2b-f8db-4d80-b950-69697596a75d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:35.251905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f7c2b-f8db-4d80-b950-69697596a75d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:35.253377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99f7c2b-f8db-4d80-b950-69697596a75d service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:35.271807 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 583/1132] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:37.454639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596192f-9f57-450e-a9f4-f2934155f229 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:37.455152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596192f-9f57-450e-a9f4-f2934155f229 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:37.455152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8596192f-9f57-450e-a9f4-f2934155f229 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:37.456450 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 550/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:37.461648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a87208-3217-4348-b142-9f83c7812623 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:37.462053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a87208-3217-4348-b142-9f83c7812623 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:37.462204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a87208-3217-4348-b142-9f83c7812623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:37.527076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-74a87208-3217-4348-b142-9f83c7812623 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:37.528265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-74a87208-3217-4348-b142-9f83c7812623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:37.557331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:37.622731 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 584/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:49:37.909275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596192f-9f57-450e-a9f4-f2934155f229 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:37.910829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8596192f-9f57-450e-a9f4-f2934155f229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:38.011282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8596192f-9f57-450e-a9f4-f2934155f229 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:38.039066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:38.083195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198de6a-37ac-4428-adb8-efbb112d9516 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:38.083722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198de6a-37ac-4428-adb8-efbb112d9516 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:38.084002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198de6a-37ac-4428-adb8-efbb112d9516 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:38.084269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198de6a-37ac-4428-adb8-efbb112d9516 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:38.085849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6198de6a-37ac-4428-adb8-efbb112d9516 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:38.190615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198de6a-37ac-4428-adb8-efbb112d9516 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:38.192023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198de6a-37ac-4428-adb8-efbb112d9516 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:38.209174 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 551/1135] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:40.393219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346709-3a72-4197-883f-683d3846b2bd None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:40.393757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346709-3a72-4197-883f-683d3846b2bd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:40.393757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a346709-3a72-4197-883f-683d3846b2bd None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:40.395102 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 585/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:40.400592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9f1ea-eb53-402e-bd44-11826a961d21 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:40.401046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9f1ea-eb53-402e-bd44-11826a961d21 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:40.401144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9f1ea-eb53-402e-bd44-11826a961d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:40.467075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e5f9f1ea-eb53-402e-bd44-11826a961d21 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:40.468270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e5f9f1ea-eb53-402e-bd44-11826a961d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:40.476769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:40.564895 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 552/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:40.847158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346709-3a72-4197-883f-683d3846b2bd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:40.848713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2a346709-3a72-4197-883f-683d3846b2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:40.951065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a346709-3a72-4197-883f-683d3846b2bd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:41.004723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:41.022440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc896498-ddc2-422c-b17c-6621e1730412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:41.023034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896498-ddc2-422c-b17c-6621e1730412 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:41.023313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896498-ddc2-422c-b17c-6621e1730412 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:41.023582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc896498-ddc2-422c-b17c-6621e1730412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:41.025155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dc896498-ddc2-422c-b17c-6621e1730412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:41.131067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc896498-ddc2-422c-b17c-6621e1730412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:41.132512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc896498-ddc2-422c-b17c-6621e1730412 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:41.149973 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 586/1138] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:43.336241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01355462-3142-484c-8cd1-ea28b147b115 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:43.336853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01355462-3142-484c-8cd1-ea28b147b115 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:43.336853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01355462-3142-484c-8cd1-ea28b147b115 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:43.338158 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 553/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:43.343709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2f47a-2ff8-41b5-8ada-1ff224040134 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:43.344128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2f47a-2ff8-41b5-8ada-1ff224040134 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:43.344228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2f47a-2ff8-41b5-8ada-1ff224040134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:43.408821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-16c2f47a-2ff8-41b5-8ada-1ff224040134 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:43.410022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-16c2f47a-2ff8-41b5-8ada-1ff224040134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:43.447419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:43.507725 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 587/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:43.798836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01355462-3142-484c-8cd1-ea28b147b115 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:43.800447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-01355462-3142-484c-8cd1-ea28b147b115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:43.903560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01355462-3142-484c-8cd1-ea28b147b115 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:43.940415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:43.975604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbc8ddc-856f-4d2f-9e8b-0602da1dea1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:43.976207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc8ddc-856f-4d2f-9e8b-0602da1dea1c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:43.976510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc8ddc-856f-4d2f-9e8b-0602da1dea1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:43.976781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbc8ddc-856f-4d2f-9e8b-0602da1dea1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:43.978339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6dbc8ddc-856f-4d2f-9e8b-0602da1dea1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:44.083788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc8ddc-856f-4d2f-9e8b-0602da1dea1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:44.085233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbc8ddc-856f-4d2f-9e8b-0602da1dea1c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:44.102866 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 554/1141] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:46.298882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dcc8b-ac6a-4092-8952-60abd66b6f93 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:46.299446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dcc8b-ac6a-4092-8952-60abd66b6f93 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:46.299446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85dcc8b-ac6a-4092-8952-60abd66b6f93 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:46.300767 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 588/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:46.306171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7146-f57c-41ac-8ff8-57880cbde63e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:46.306575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7146-f57c-41ac-8ff8-57880cbde63e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:46.306715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d7146-f57c-41ac-8ff8-57880cbde63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:46.372715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ff0d7146-f57c-41ac-8ff8-57880cbde63e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:46.373893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ff0d7146-f57c-41ac-8ff8-57880cbde63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:46.393039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:46.469163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 555/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:46.757852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dcc8b-ac6a-4092-8952-60abd66b6f93 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:46.759370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f85dcc8b-ac6a-4092-8952-60abd66b6f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:46.862974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85dcc8b-ac6a-4092-8952-60abd66b6f93 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:46.927979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:46.934493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73b788-01ba-4852-9751-027fc1d7db19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:46.935121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73b788-01ba-4852-9751-027fc1d7db19 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:46.935393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73b788-01ba-4852-9751-027fc1d7db19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:46.935659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73b788-01ba-4852-9751-027fc1d7db19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:46.937293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5b73b788-01ba-4852-9751-027fc1d7db19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:47.041796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73b788-01ba-4852-9751-027fc1d7db19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:47.043159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73b788-01ba-4852-9751-027fc1d7db19 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:47.061271 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 589/1144] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:49.201438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf277-4922-4cc0-b94d-dcf9733c302f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:49.201999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf277-4922-4cc0-b94d-dcf9733c302f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:49.201999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07abf277-4922-4cc0-b94d-dcf9733c302f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:49.203319 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 556/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:49:49.208523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00d565-8c43-4a33-9899-30882a54efd2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:49.208941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00d565-8c43-4a33-9899-30882a54efd2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:49.209067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa00d565-8c43-4a33-9899-30882a54efd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:49.273657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-aa00d565-8c43-4a33-9899-30882a54efd2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:49:49.274842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-aa00d565-8c43-4a33-9899-30882a54efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:49.322860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:49.371221 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 590/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:49:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:49:49.649801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abf277-4922-4cc0-b94d-dcf9733c302f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:49:49.651674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-07abf277-4922-4cc0-b94d-dcf9733c302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:49.753011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07abf277-4922-4cc0-b94d-dcf9733c302f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:49:49.800169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:49.825729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb07ad-e238-4eee-8ecc-d0c731dc5d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:49:49.826331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb07ad-e238-4eee-8ecc-d0c731dc5d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:49:49.826604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb07ad-e238-4eee-8ecc-d0c731dc5d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:49:49.826870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb07ad-e238-4eee-8ecc-d0c731dc5d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:49:49.828949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ffb07ad-e238-4eee-8ecc-d0c731dc5d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:49:49.937544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb07ad-e238-4eee-8ecc-d0c731dc5d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:49:49.938882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb07ad-e238-4eee-8ecc-d0c731dc5d0c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:49:49.956402 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 557/1147] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:49:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:49:59.369697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=22 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:49:59.954164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=66 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:09.771541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57dfc0-08f6-4e6f-9e1d-e14ed6f107c6 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:09.772134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57dfc0-08f6-4e6f-9e1d-e14ed6f107c6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:09.772134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b57dfc0-08f6-4e6f-9e1d-e14ed6f107c6 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:09.773501 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 591/1148] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:50:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:50:09.778630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7a5c9-705d-4c4e-a6c6-f94eaf75c4aa None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:09.779614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7a5c9-705d-4c4e-a6c6-f94eaf75c4aa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:09.779731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd7a5c9-705d-4c4e-a6c6-f94eaf75c4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:09.845866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0fd7a5c9-705d-4c4e-a6c6-f94eaf75c4aa None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:09.847046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0fd7a5c9-705d-4c4e-a6c6-f94eaf75c4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:09.942236 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 558/1149] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:50:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:10.238297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516c14-c755-4ba6-b3e0-f78d95fd2229 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:10.238582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516c14-c755-4ba6-b3e0-f78d95fd2229 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:10.238855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516c14-c755-4ba6-b3e0-f78d95fd2229 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:10.240193 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 592/1150] 10.222.0.39 () {54 vars in 962 bytes} [Sun Oct 23 00:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:50:10.245677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d065-2654-4486-946b-4077f4d2fa40 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:10.245968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d065-2654-4486-946b-4077f4d2fa40 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:10.246263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6452d065-2654-4486-946b-4077f4d2fa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:10.321562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6452d065-2654-4486-946b-4077f4d2fa40 None None] MFA Rules not processed for user `b5429182e2fa4e2aa485fc084e32aafd`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:10.322755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6452d065-2654-4486-946b-4077f4d2fa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:10.343412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:10.398222 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 559/1151] 10.222.0.39 () {58 vars in 1059 bytes} [Sun Oct 23 00:50:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:10.405973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4516c14-c755-4ba6-b3e0-f78d95fd2229 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:50:10.407535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f4516c14-c755-4ba6-b3e0-f78d95fd2229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:10.510797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4516c14-c755-4ba6-b3e0-f78d95fd2229 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:50:10.583747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f47d33-42f6-40f5-a493-83becd69febb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:50:10.584386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47d33-42f6-40f5-a493-83becd69febb service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:10.584666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47d33-42f6-40f5-a493-83becd69febb service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:10.584939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f47d33-42f6-40f5-a493-83becd69febb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:10.586501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-92f47d33-42f6-40f5-a493-83becd69febb service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:10.639028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:10.692967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f47d33-42f6-40f5-a493-83becd69febb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:50:10.694396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f47d33-42f6-40f5-a493-83becd69febb service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:50:10.712273 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 593/1152] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:50:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:50:12.925961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1cf48-5d1e-4fc5-95ce-bbfd65e267e9 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:12.926522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1cf48-5d1e-4fc5-95ce-bbfd65e267e9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:12.926522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1cf48-5d1e-4fc5-95ce-bbfd65e267e9 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:12.927823 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 560/1153] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:50:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:50:12.933222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76c9f1-7672-4e04-81cc-dcffb8e2d4ea None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:12.933659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76c9f1-7672-4e04-81cc-dcffb8e2d4ea None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:12.933753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b76c9f1-7672-4e04-81cc-dcffb8e2d4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:12.998484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6b76c9f1-7672-4e04-81cc-dcffb8e2d4ea None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:12.999673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6b76c9f1-7672-4e04-81cc-dcffb8e2d4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:13.092457 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 594/1154] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:50:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:13.362208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1cf48-5d1e-4fc5-95ce-bbfd65e267e9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:50:13.363635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-afb1cf48-5d1e-4fc5-95ce-bbfd65e267e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:13.484963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1cf48-5d1e-4fc5-95ce-bbfd65e267e9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:50:13.536121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:13.579966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcb38ee-cf55-4898-bc02-68c9b10c1c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:50:13.580593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb38ee-cf55-4898-bc02-68c9b10c1c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:13.580867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb38ee-cf55-4898-bc02-68c9b10c1c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:13.581132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcb38ee-cf55-4898-bc02-68c9b10c1c89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:13.581757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb38ee-cf55-4898-bc02-68c9b10c1c89 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:50:13.582837 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-9bcb38ee-cf55-4898-bc02-68c9b10c1c89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:50:13.582837 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:50:13.582837 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:50:13.583952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcb38ee-cf55-4898-bc02-68c9b10c1c89 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:50:13.602232 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 561/1155] 10.222.0.39 () {56 vars in 1108 bytes} [Sun Oct 23 00:50:13 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:50:15.782127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90372d1e-dfa3-47c3-80ea-0cdc757ca47d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:15.782693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90372d1e-dfa3-47c3-80ea-0cdc757ca47d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:15.782693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90372d1e-dfa3-47c3-80ea-0cdc757ca47d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:15.784049 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 595/1156] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:50:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:50:15.789592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed38db9-a971-483c-9b32-1aa7a03e8e0b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:15.789967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed38db9-a971-483c-9b32-1aa7a03e8e0b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:15.790166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed38db9-a971-483c-9b32-1aa7a03e8e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:15.856935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9ed38db9-a971-483c-9b32-1aa7a03e8e0b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:15.858124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9ed38db9-a971-483c-9b32-1aa7a03e8e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:15.952544 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 562/1157] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:16.245468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90372d1e-dfa3-47c3-80ea-0cdc757ca47d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:50:16.247547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-90372d1e-dfa3-47c3-80ea-0cdc757ca47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:16.257634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:16.393798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90372d1e-dfa3-47c3-80ea-0cdc757ca47d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:50:16.474082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:16.507449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58570fa2-7ed5-4976-b575-a2facf466728 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:50:16.508019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570fa2-7ed5-4976-b575-a2facf466728 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:16.508327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570fa2-7ed5-4976-b575-a2facf466728 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:16.508590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58570fa2-7ed5-4976-b575-a2facf466728 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:16.510441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-58570fa2-7ed5-4976-b575-a2facf466728 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:16.596602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58570fa2-7ed5-4976-b575-a2facf466728 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:50:16.598175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58570fa2-7ed5-4976-b575-a2facf466728 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:50:16.615912 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 596/1158] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:50:18.930325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1391-aba6-450d-acc2-dec1e0495f12 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:18.930906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1391-aba6-450d-acc2-dec1e0495f12 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:18.930906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0f1391-aba6-450d-acc2-dec1e0495f12 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:18.932234 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 563/1159] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:50:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:50:18.938418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f3d19-9dec-4feb-8818-be030245fd21 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:18.938865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f3d19-9dec-4feb-8818-be030245fd21 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:18.939067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827f3d19-9dec-4feb-8818-be030245fd21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:19.003797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-827f3d19-9dec-4feb-8818-be030245fd21 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:19.005016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-827f3d19-9dec-4feb-8818-be030245fd21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:19.049515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:19.098731 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 597/1160] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:50:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:19.386379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f1391-aba6-450d-acc2-dec1e0495f12 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:50:19.387951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dd0f1391-aba6-450d-acc2-dec1e0495f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:19.411421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:19.489388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0f1391-aba6-450d-acc2-dec1e0495f12 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:50:19.561246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6571af-c0c6-4777-9fc7-6915faa5328a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:50:19.561846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6571af-c0c6-4777-9fc7-6915faa5328a service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:19.562125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6571af-c0c6-4777-9fc7-6915faa5328a service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:19.562393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6571af-c0c6-4777-9fc7-6915faa5328a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:19.563959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b6571af-c0c6-4777-9fc7-6915faa5328a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:19.635638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:19.670393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6571af-c0c6-4777-9fc7-6915faa5328a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:50:19.671816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6571af-c0c6-4777-9fc7-6915faa5328a service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:50:19.689773 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 564/1161] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:50:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:50:21.868429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4571fb-b2f2-4479-a679-f68e0bf6b5fa None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:21.868933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4571fb-b2f2-4479-a679-f68e0bf6b5fa None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:21.869048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4571fb-b2f2-4479-a679-f68e0bf6b5fa None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:21.870297 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 598/1162] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:50:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:50:21.875551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e399018-1824-44ac-a32f-fd515591d2ec None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:21.875959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e399018-1824-44ac-a32f-fd515591d2ec None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:21.876098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e399018-1824-44ac-a32f-fd515591d2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:21.942422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0e399018-1824-44ac-a32f-fd515591d2ec None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:21.943641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e399018-1824-44ac-a32f-fd515591d2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:22.039030 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 565/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:22.325882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4571fb-b2f2-4479-a679-f68e0bf6b5fa None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:50:22.327384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9f4571fb-b2f2-4479-a679-f68e0bf6b5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:22.429664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4571fb-b2f2-4479-a679-f68e0bf6b5fa None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:50:22.499481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:22.500938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456b0cf4-9f1e-4559-bce3-47ed691fbf81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:50:22.501545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0cf4-9f1e-4559-bce3-47ed691fbf81 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:22.502003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0cf4-9f1e-4559-bce3-47ed691fbf81 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:22.502489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456b0cf4-9f1e-4559-bce3-47ed691fbf81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:22.504263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-456b0cf4-9f1e-4559-bce3-47ed691fbf81 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:22.611654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0cf4-9f1e-4559-bce3-47ed691fbf81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:50:22.613027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456b0cf4-9f1e-4559-bce3-47ed691fbf81 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:50:22.630426 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 599/1164] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:50:24.858317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15b4ab-fe45-469b-a032-8d30b3a07700 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:24.858779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15b4ab-fe45-469b-a032-8d30b3a07700 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:24.858874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f15b4ab-fe45-469b-a032-8d30b3a07700 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:24.860192 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 566/1165] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:50:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:50:24.865477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df029f7-733d-4f73-a639-96b3663179f8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:24.865903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df029f7-733d-4f73-a639-96b3663179f8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:24.866016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df029f7-733d-4f73-a639-96b3663179f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:24.933013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9df029f7-733d-4f73-a639-96b3663179f8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:24.934873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9df029f7-733d-4f73-a639-96b3663179f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:24.986258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:25.029265 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 600/1166] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:25.323476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15b4ab-fe45-469b-a032-8d30b3a07700 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:50:25.325652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6f15b4ab-fe45-469b-a032-8d30b3a07700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:25.360480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:25.427018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f15b4ab-fe45-469b-a032-8d30b3a07700 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:50:25.499643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41553192-44e9-438d-98ba-eca884b9aa62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:50:25.500248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41553192-44e9-438d-98ba-eca884b9aa62 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:25.500559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41553192-44e9-438d-98ba-eca884b9aa62 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:25.500835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41553192-44e9-438d-98ba-eca884b9aa62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:25.502408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-41553192-44e9-438d-98ba-eca884b9aa62 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:25.582676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:25.611261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41553192-44e9-438d-98ba-eca884b9aa62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:50:25.612724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41553192-44e9-438d-98ba-eca884b9aa62 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:50:25.630152 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 567/1167] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:50:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:50:27.967998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773df11-bfa3-4ade-b961-363a7cc1e591 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:27.968610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773df11-bfa3-4ade-b961-363a7cc1e591 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:27.968610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773df11-bfa3-4ade-b961-363a7cc1e591 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:27.969836 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 601/1168] 10.222.0.39 () {54 vars in 927 bytes} [Sun Oct 23 00:50:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:50:27.975171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213dcf-d786-4f1f-bb5d-a1af30958bd3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:27.975596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213dcf-d786-4f1f-bb5d-a1af30958bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:27.975700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213dcf-d786-4f1f-bb5d-a1af30958bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:28.042269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-26213dcf-d786-4f1f-bb5d-a1af30958bd3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:50:28.043995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-26213dcf-d786-4f1f-bb5d-a1af30958bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:28.137579 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 568/1169] 10.222.0.39 () {58 vars in 1024 bytes} [Sun Oct 23 00:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:50:28.427190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773df11-bfa3-4ade-b961-363a7cc1e591 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:50:28.428736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5773df11-bfa3-4ade-b961-363a7cc1e591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:28.532615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773df11-bfa3-4ade-b961-363a7cc1e591 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:50:28.604291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38e0d-e2a9-4867-ab4b-fba3581ed2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:50:28.604905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38e0d-e2a9-4867-ab4b-fba3581ed2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:50:28.605203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38e0d-e2a9-4867-ab4b-fba3581ed2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:50:28.605502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38e0d-e2a9-4867-ab4b-fba3581ed2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:50:28.607053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5dc38e0d-e2a9-4867-ab4b-fba3581ed2c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:50:28.615212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:28.712668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc38e0d-e2a9-4867-ab4b-fba3581ed2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:50:28.714014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc38e0d-e2a9-4867-ab4b-fba3581ed2c7 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:50:28.732471 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 602/1170] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:50:38.136576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=82 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:50:38.730247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=50 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:43.155445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:43.156003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:43.156003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:43.230516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:51:43.231698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:43.326418 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 569/1171] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:51:43.334261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4aa4f-bc4f-49a5-8690-69fcdde1c534 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:43.335435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4aa4f-bc4f-49a5-8690-69fcdde1c534 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:43.335541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc4aa4f-bc4f-49a5-8690-69fcdde1c534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:43.409189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cfc4aa4f-bc4f-49a5-8690-69fcdde1c534 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:51:43.410297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cfc4aa4f-bc4f-49a5-8690-69fcdde1c534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:43.503959 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 603/1172] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:51:43.509379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:43.510562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:43.521974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e988f6-9f61-42b6-a781-5152d6f55bf9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:43.583467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:43.616362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:43.616963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:43.617240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:43.617510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:43.618144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:51:43.618440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:43.619477 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:51:43.619477 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:51:43.619477 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:51:43.620368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:43.631622 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 570/1173] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:51:43.641495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4aa4f-bc4f-49a5-8690-69fcdde1c534 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:43.651259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc4aa4f-bc4f-49a5-8690-69fcdde1c534 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:43.711851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:43.743846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:43.744473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:43.744750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:43.745017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:43.745642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:43.746727 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:51:43.746727 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:51:43.746727 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:51:43.747928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:43.803308 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 604/1174] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:51:43 2022] POST /identity/v3/projects => generated 379 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:51:43.807676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:43.809266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:43.844211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f9e213-e202-46ba-ac3b-70356e9fa523 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:43.936448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:43.937050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:43.937329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:43.937595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:43.938211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:43.939203 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:51:43.939203 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:51:43.939203 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:51:43.940427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:43.981176 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 571/1175] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:51:43 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:51:43.985325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:43.986846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:44.047028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8ff09-a792-416c-a2ad-bdec2a722359 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:44.128217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:44.140132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:44.140769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:44.141048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:44.141318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:44.141955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:44.143571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:44.155829 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 605/1176] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:51:44.160032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:44.161585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:44.172810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504375d-d527-42c0-817b-0437396ec37d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:44.198191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:44.266803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeca811d-a678-41ec-bfc7-a14d42f6054b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:44.267357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca811d-a678-41ec-bfc7-a14d42f6054b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:44.267615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca811d-a678-41ec-bfc7-a14d42f6054b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:44.267876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeca811d-a678-41ec-bfc7-a14d42f6054b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:44.268539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca811d-a678-41ec-bfc7-a14d42f6054b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:44.270107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeca811d-a678-41ec-bfc7-a14d42f6054b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:44.282132 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 572/1177] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:51:44 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:51:44.286396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:44.288446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:44.321727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c1e39-4cbe-407f-9f56-f3ece7499d8e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:44.412071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d8517-3ad5-40c5-a825-6fc858873451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:44.412797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d8517-3ad5-40c5-a825-6fc858873451 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:44.413072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d8517-3ad5-40c5-a825-6fc858873451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:44.413342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d8517-3ad5-40c5-a825-6fc858873451 admin admin] PATH_INFO: `/v3/projects/826d7d73699a4d47ae776d5bf62993bc/users/ba5f4e2ba28c4df68a90616ea4af5dae/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:44.436424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d8517-3ad5-40c5-a825-6fc858873451 admin admin] RBAC: Authorizing `identity:create_grant(project_id=826d7d73699a4d47ae776d5bf62993bc, user_id=ba5f4e2ba28c4df68a90616ea4af5dae, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:44.439042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d8517-3ad5-40c5-a825-6fc858873451 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:44.463308 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 606/1178] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:51:44 2022] PUT /identity/v3/projects/826d7d73699a4d47ae776d5bf62993bc/users/ba5f4e2ba28c4df68a90616ea4af5dae/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:51:44.474129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d89ca8-22f1-4808-9925-a79ff2e83aac None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:44.474411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d89ca8-22f1-4808-9925-a79ff2e83aac None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:44.474760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d89ca8-22f1-4808-9925-a79ff2e83aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:44.531289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-77d89ca8-22f1-4808-9925-a79ff2e83aac None None] MFA Rules not processed for user `ba5f4e2ba28c4df68a90616ea4af5dae`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:51:44.532435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-77d89ca8-22f1-4808-9925-a79ff2e83aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:44.606501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 573/1179] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:51:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:51:44.619755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92b798-9a5a-4c9e-8229-47046e3eb172 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:44.620039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92b798-9a5a-4c9e-8229-47046e3eb172 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:44.620326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92b798-9a5a-4c9e-8229-47046e3eb172 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:44.621668 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 607/1180] 10.222.0.39 () {54 vars in 963 bytes} [Sun Oct 23 00:51:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:51:44.626513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc989eb-91fc-4c57-8691-30c6351fe20b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:44.626795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc989eb-91fc-4c57-8691-30c6351fe20b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:44.627069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc989eb-91fc-4c57-8691-30c6351fe20b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:44.658397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:44.703101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5cc989eb-91fc-4c57-8691-30c6351fe20b None None] MFA Rules not processed for user `9075ccb9e3cd46cc9e77caa4060e3c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:51:44.704294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5cc989eb-91fc-4c57-8691-30c6351fe20b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:44.777801 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 574/1181] 10.222.0.39 () {58 vars in 1060 bytes} [Sun Oct 23 00:51:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:51:44.785494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92b798-9a5a-4c9e-8229-47046e3eb172 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:44.786915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5a92b798-9a5a-4c9e-8229-47046e3eb172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:44.836782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:44.890221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92b798-9a5a-4c9e-8229-47046e3eb172 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:44.963609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:44.964207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:44.964514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:44.964781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:44.966343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:45.062191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:45.063609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:45.067008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:45.081615 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 608/1182] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:51:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:51:45.126895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccb826-0b46-4000-9ba1-244de010ce49 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:45.127186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccb826-0b46-4000-9ba1-244de010ce49 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:45.127464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccb826-0b46-4000-9ba1-244de010ce49 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:45.128770 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 575/1183] 10.222.0.39 () {52 vars in 830 bytes} [Sun Oct 23 00:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:51:45.132541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:45.133761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:45.230683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39292c-5c97-42f1-a935-5f02cd15b6a1 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:45.329142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:45.329702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:45.329980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:45.330248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] PATH_INFO: `/v3/users/ba5f4e2ba28c4df68a90616ea4af5dae` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:45.343613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba5f4e2ba28c4df68a90616ea4af5dae)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:45.344663 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:51:45.344663 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:51:45.344663 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:51:45.345882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:45.356224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:45.409906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba5f4e2ba28c4df68a90616ea4af5dae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:45.424057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba5f4e2ba28c4df68a90616ea4af5dae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:45.433869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba5f4e2ba28c4df68a90616ea4af5dae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:45.456755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba5f4e2ba28c4df68a90616ea4af5dae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:45.457291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba5f4e2ba28c4df68a90616ea4af5dae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:45.463613 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 609/1184] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:51:45 2022] DELETE /identity/v3/users/ba5f4e2ba28c4df68a90616ea4af5dae => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:51:45.467933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:45.468410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:45.468507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:45.527030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:51:45.528233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:45.602808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:45.615291 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 576/1185] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:51:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:51:45.624868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:45.626153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:45.726346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d0142-40a8-4058-87b4-05683f862534 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:45.780389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:45.800547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5a62bd-602e-446a-ab8a-7cf30b87b6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:45.801182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a62bd-602e-446a-ab8a-7cf30b87b6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:45.801459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a62bd-602e-446a-ab8a-7cf30b87b6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:45.801728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5a62bd-602e-446a-ab8a-7cf30b87b6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:45.803320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6e5a62bd-602e-446a-ab8a-7cf30b87b6c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:45.910384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a62bd-602e-446a-ab8a-7cf30b87b6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:45.911743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5a62bd-602e-446a-ab8a-7cf30b87b6c2 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:45.930769 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 610/1186] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:51:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:51:46.284797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:51:46.286017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:51:46.298105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd249e9f-09af-4584-b247-d8395534afc8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:51:46.396439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:51:46.397129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:51:46.397407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:51:46.397683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] PATH_INFO: `/v3/projects/826d7d73699a4d47ae776d5bf62993bc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:51:46.407886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] RBAC: Authorizing `identity:delete_project(project_id=826d7d73699a4d47ae776d5bf62993bc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:51:46.408945 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:51:46.408945 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:51:46.408945 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:51:46.410165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:51:46.469010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '826d7d73699a4d47ae776d5bf62993bc', 'request_id': 'req-3aa7d52f-c24a-479c-958d-5c03ce52161a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:46.480105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '826d7d73699a4d47ae776d5bf62993bc', 'request_id': 'req-3aa7d52f-c24a-479c-958d-5c03ce52161a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:46.486224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '826d7d73699a4d47ae776d5bf62993bc', 'request_id': 'req-3aa7d52f-c24a-479c-958d-5c03ce52161a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:46.517717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] The token cache is being invalidate because project 826d7d73699a4d47ae776d5bf62993bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:51:46.518003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:46.518426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3aa7d52f-c24a-479c-958d-5c03ce52161a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:51:46.519857 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 577/1187] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:51:46 2022] DELETE /identity/v3/projects/826d7d73699a4d47ae776d5bf62993bc => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:51:55.928418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=62 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:51:56.515822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:53:16.484924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02c45-6954-4133-993d-52addda6d26a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:16.485524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02c45-6954-4133-993d-52addda6d26a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:16.485524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd02c45-6954-4133-993d-52addda6d26a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:16.486797 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 611/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:16.491726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018fec0-7293-42eb-954c-8073d19b3614 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:16.492169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018fec0-7293-42eb-954c-8073d19b3614 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:16.492279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018fec0-7293-42eb-954c-8073d19b3614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:16.574679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9018fec0-7293-42eb-954c-8073d19b3614 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:16.575872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9018fec0-7293-42eb-954c-8073d19b3614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:16.676705 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 578/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:16.834191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd02c45-6954-4133-993d-52addda6d26a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:16.836352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-abd02c45-6954-4133-993d-52addda6d26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:16.951634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd02c45-6954-4133-993d-52addda6d26a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:16.957095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e07b0-9bfc-4332-b254-0213962b0191 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:16.957693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e07b0-9bfc-4332-b254-0213962b0191 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:16.957974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e07b0-9bfc-4332-b254-0213962b0191 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:16.958776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e07b0-9bfc-4332-b254-0213962b0191 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:16.966754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e07b0-9bfc-4332-b254-0213962b0191 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:16.968375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e07b0-9bfc-4332-b254-0213962b0191 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:16.978057 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 612/1190] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:53:18.047160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975f83-f34e-4f80-ae0a-65122df712f8 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:18.047697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975f83-f34e-4f80-ae0a-65122df712f8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:18.047697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e975f83-f34e-4f80-ae0a-65122df712f8 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:18.049042 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 579/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:18.053953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225997b6-f725-4ff4-80ba-e320b5196bca None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:18.054377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225997b6-f725-4ff4-80ba-e320b5196bca None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:18.054487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225997b6-f725-4ff4-80ba-e320b5196bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:18.074892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-225997b6-f725-4ff4-80ba-e320b5196bca None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:18.076085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-225997b6-f725-4ff4-80ba-e320b5196bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:18.084397 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 613/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:53:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:18.235174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e975f83-f34e-4f80-ae0a-65122df712f8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:18.249558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e975f83-f34e-4f80-ae0a-65122df712f8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:18.255678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bae3a3-ccf0-437b-bc35-c3dbfc360741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:18.256271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bae3a3-ccf0-437b-bc35-c3dbfc360741 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:18.256577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bae3a3-ccf0-437b-bc35-c3dbfc360741 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:18.256845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bae3a3-ccf0-437b-bc35-c3dbfc360741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:18.264577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bae3a3-ccf0-437b-bc35-c3dbfc360741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:18.266155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bae3a3-ccf0-437b-bc35-c3dbfc360741 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:18.275170 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 580/1193] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:18.452540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e5a39-3caf-48fe-9890-77481772aff0 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:18.452824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e5a39-3caf-48fe-9890-77481772aff0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:18.453094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05e5a39-3caf-48fe-9890-77481772aff0 None None] PATH_INFO: `/v3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:18.454380 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 614/1194] 10.222.0.43 () {54 vars in 925 bytes} [Sun Oct 23 00:53:18 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:53:18.460073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958217-a3e5-4e1c-9007-50c20941d3f1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:18.460378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958217-a3e5-4e1c-9007-50c20941d3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:18.460653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958217-a3e5-4e1c-9007-50c20941d3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:18.518802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7d958217-a3e5-4e1c-9007-50c20941d3f1 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:18.520126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7d958217-a3e5-4e1c-9007-50c20941d3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:18.537744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:53:18.590525 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 581/1195] 10.222.0.43 () {58 vars in 1015 bytes} [Sun Oct 23 00:53:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:53:22.225431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85645596-75df-4b5a-9783-750a6a14a755 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:22.225935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85645596-75df-4b5a-9783-750a6a14a755 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:22.226042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85645596-75df-4b5a-9783-750a6a14a755 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:22.227384 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 615/1196] 10.222.0.39 () {54 vars in 961 bytes} [Sun Oct 23 00:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:22.233450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d99730-ac8f-48cf-8cdf-947db89a4d47 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:22.233930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d99730-ac8f-48cf-8cdf-947db89a4d47 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:22.234015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d99730-ac8f-48cf-8cdf-947db89a4d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:22.258231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-85d99730-ac8f-48cf-8cdf-947db89a4d47 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:22.259414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-85d99730-ac8f-48cf-8cdf-947db89a4d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:22.267747 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 582/1197] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Oct 23 00:53:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:22.275397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85645596-75df-4b5a-9783-750a6a14a755 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:22.288556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85645596-75df-4b5a-9783-750a6a14a755 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:22.293549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31328b2-023e-4e73-b0f5-839b4a7df514 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:22.294146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31328b2-023e-4e73-b0f5-839b4a7df514 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:22.294424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31328b2-023e-4e73-b0f5-839b4a7df514 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:22.294692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31328b2-023e-4e73-b0f5-839b4a7df514 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:22.303541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31328b2-023e-4e73-b0f5-839b4a7df514 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:22.305160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31328b2-023e-4e73-b0f5-839b4a7df514 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:22.314696 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 616/1198] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:29.726928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71493-2847-4faa-9e1c-c6c316365e4f None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:29.727490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71493-2847-4faa-9e1c-c6c316365e4f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:29.727490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b71493-2847-4faa-9e1c-c6c316365e4f None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:29.728865 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 583/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:29.733654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342aa6-43b3-4fa2-ab6f-8b5ccb749595 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:29.734083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342aa6-43b3-4fa2-ab6f-8b5ccb749595 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:29.734193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342aa6-43b3-4fa2-ab6f-8b5ccb749595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:29.753086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ff342aa6-43b3-4fa2-ab6f-8b5ccb749595 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:29.754273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ff342aa6-43b3-4fa2-ab6f-8b5ccb749595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:29.762107 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 617/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:53:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:29.914985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b71493-2847-4faa-9e1c-c6c316365e4f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:29.929014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b71493-2847-4faa-9e1c-c6c316365e4f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:29.934214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac04ad83-af4e-48ae-86e0-5cc272ffdfb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:29.934799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04ad83-af4e-48ae-86e0-5cc272ffdfb1 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:29.935073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04ad83-af4e-48ae-86e0-5cc272ffdfb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:29.935338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac04ad83-af4e-48ae-86e0-5cc272ffdfb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:29.943712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04ad83-af4e-48ae-86e0-5cc272ffdfb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:29.945299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac04ad83-af4e-48ae-86e0-5cc272ffdfb1 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:29.954318 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 584/1201] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:31.002237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b10ba5-7c92-424e-91d4-3d4b18c09011 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:31.002802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b10ba5-7c92-424e-91d4-3d4b18c09011 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:31.002802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b10ba5-7c92-424e-91d4-3d4b18c09011 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:31.004168 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 618/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 00:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:31.009315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5323a-8717-4032-9142-2f6d678abca1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:31.009726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5323a-8717-4032-9142-2f6d678abca1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:31.009859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5323a-8717-4032-9142-2f6d678abca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:31.029399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4fe5323a-8717-4032-9142-2f6d678abca1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:31.030576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4fe5323a-8717-4032-9142-2f6d678abca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:31.038912 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 585/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 00:53:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:31.190083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b10ba5-7c92-424e-91d4-3d4b18c09011 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:31.204075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b10ba5-7c92-424e-91d4-3d4b18c09011 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:31.209982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a048481-c6ce-4122-bab5-dfd051a9a1fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:31.210571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a048481-c6ce-4122-bab5-dfd051a9a1fd service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:31.210854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a048481-c6ce-4122-bab5-dfd051a9a1fd service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:31.211147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a048481-c6ce-4122-bab5-dfd051a9a1fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:31.218941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a048481-c6ce-4122-bab5-dfd051a9a1fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:31.220606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a048481-c6ce-4122-bab5-dfd051a9a1fd service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:31.230394 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 619/1204] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:35.335076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb88f1-f860-4627-a279-ec7a3e33a7fc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:35.335651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb88f1-f860-4627-a279-ec7a3e33a7fc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:35.335651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb88f1-f860-4627-a279-ec7a3e33a7fc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:35.337013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 586/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:35.342134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160241f2-2b05-4706-9795-24e5ab1094f3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:35.342538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160241f2-2b05-4706-9795-24e5ab1094f3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:35.342682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160241f2-2b05-4706-9795-24e5ab1094f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:35.362924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-160241f2-2b05-4706-9795-24e5ab1094f3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:35.364105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-160241f2-2b05-4706-9795-24e5ab1094f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:35.372455 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 620/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:53:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:35.658861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb88f1-f860-4627-a279-ec7a3e33a7fc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:35.660618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d0fb88f1-f860-4627-a279-ec7a3e33a7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:35.736319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb88f1-f860-4627-a279-ec7a3e33a7fc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:35.740592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ec398-1bb7-4673-9cba-18673d631a08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:35.741212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ec398-1bb7-4673-9cba-18673d631a08 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:35.741490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ec398-1bb7-4673-9cba-18673d631a08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:35.741761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ec398-1bb7-4673-9cba-18673d631a08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:35.750678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ec398-1bb7-4673-9cba-18673d631a08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:35.752107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ec398-1bb7-4673-9cba-18673d631a08 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:35.762155 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 587/1207] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:37.872324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa509b9-ecb8-4863-9926-7517bded3ef9 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:37.872849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa509b9-ecb8-4863-9926-7517bded3ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:37.872849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa509b9-ecb8-4863-9926-7517bded3ef9 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:37.874189 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 621/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:37.879590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b847cc-74f7-4ada-9aca-d0d099f655a2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:37.880007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b847cc-74f7-4ada-9aca-d0d099f655a2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:37.880124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b847cc-74f7-4ada-9aca-d0d099f655a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:37.899933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-13b847cc-74f7-4ada-9aca-d0d099f655a2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:37.901172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-13b847cc-74f7-4ada-9aca-d0d099f655a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:37.908870 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 588/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:53:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:38.192941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa509b9-ecb8-4863-9926-7517bded3ef9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:38.203395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa509b9-ecb8-4863-9926-7517bded3ef9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:38.206823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07a5bc7-2032-40d7-9a38-c9a263cc3020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:38.207414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a5bc7-2032-40d7-9a38-c9a263cc3020 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:38.207679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a5bc7-2032-40d7-9a38-c9a263cc3020 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:38.208524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07a5bc7-2032-40d7-9a38-c9a263cc3020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:38.217068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a5bc7-2032-40d7-9a38-c9a263cc3020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:38.218469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07a5bc7-2032-40d7-9a38-c9a263cc3020 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:38.227475 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 622/1210] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:41.885431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c228bac-8a11-4908-b41b-2d75a6ba144c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:41.885950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c228bac-8a11-4908-b41b-2d75a6ba144c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:41.885950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c228bac-8a11-4908-b41b-2d75a6ba144c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:41.887295 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 589/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:41.892660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac50401-27b5-4c71-9aa7-50667cd2758d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:41.893062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac50401-27b5-4c71-9aa7-50667cd2758d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:41.893207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac50401-27b5-4c71-9aa7-50667cd2758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:41.912448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8ac50401-27b5-4c71-9aa7-50667cd2758d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:41.913628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ac50401-27b5-4c71-9aa7-50667cd2758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:41.921536 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 623/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:53:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:42.220710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c228bac-8a11-4908-b41b-2d75a6ba144c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:42.222392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6c228bac-8a11-4908-b41b-2d75a6ba144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:42.287793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:53:42.301564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c228bac-8a11-4908-b41b-2d75a6ba144c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:42.305059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303997aa-8125-4341-9ca3-6c3290120ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:42.305663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303997aa-8125-4341-9ca3-6c3290120ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:42.305941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303997aa-8125-4341-9ca3-6c3290120ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:42.306210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303997aa-8125-4341-9ca3-6c3290120ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:42.314883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303997aa-8125-4341-9ca3-6c3290120ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:42.316310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303997aa-8125-4341-9ca3-6c3290120ca2 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:42.325599 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 590/1213] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:44.488728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b5596-3ec4-4262-a3a3-6b197c49ad8a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:44.489852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b5596-3ec4-4262-a3a3-6b197c49ad8a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:44.489852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b5596-3ec4-4262-a3a3-6b197c49ad8a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:44.491153 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 624/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 00:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:53:44.496493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42cf7f-a429-468a-ad7f-7803691bc1b9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:44.496885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42cf7f-a429-468a-ad7f-7803691bc1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:44.497068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42cf7f-a429-468a-ad7f-7803691bc1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:44.518007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-be42cf7f-a429-468a-ad7f-7803691bc1b9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:53:44.519191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-be42cf7f-a429-468a-ad7f-7803691bc1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:44.528003 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 591/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 00:53:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:53:44.825550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b5596-3ec4-4262-a3a3-6b197c49ad8a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:53:44.827182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-110b5596-3ec4-4262-a3a3-6b197c49ad8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:53:44.840357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b5596-3ec4-4262-a3a3-6b197c49ad8a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:53:44.843731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90a527-635e-4978-8fd5-dcd3813fc7d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:53:44.844345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a527-635e-4978-8fd5-dcd3813fc7d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:53:44.844622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a527-635e-4978-8fd5-dcd3813fc7d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:53:44.844886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90a527-635e-4978-8fd5-dcd3813fc7d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:53:44.853877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a527-635e-4978-8fd5-dcd3813fc7d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:53:44.855280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90a527-635e-4978-8fd5-dcd3813fc7d1 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:53:44.864284 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 625/1216] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:53:54.513114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=11 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:53:54.860251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:16.849594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b6a95-11e9-4206-a51f-56da2d26a4b7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:16.849594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b6a95-11e9-4206-a51f-56da2d26a4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:16.849594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b6a95-11e9-4206-a51f-56da2d26a4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:16.874332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f84b6a95-11e9-4206-a51f-56da2d26a4b7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:16.877994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f84b6a95-11e9-4206-a51f-56da2d26a4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:16.886932 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 592/1217] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:16.897607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd15ee-cc57-43fb-9fbf-96a28b8962bd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:16.900641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd15ee-cc57-43fb-9fbf-96a28b8962bd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:16.900641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbd15ee-cc57-43fb-9fbf-96a28b8962bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:16.933712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4dbd15ee-cc57-43fb-9fbf-96a28b8962bd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:16.934946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4dbd15ee-cc57-43fb-9fbf-96a28b8962bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:16.944500 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 626/1218] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:16.949357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b6a95-11e9-4206-a51f-56da2d26a4b7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:16.966010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b6a95-11e9-4206-a51f-56da2d26a4b7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:16.975076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80ceee-ebce-4198-a831-b8ff3f11fc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:16.975715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80ceee-ebce-4198-a831-b8ff3f11fc56 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:16.975996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80ceee-ebce-4198-a831-b8ff3f11fc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:16.976265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80ceee-ebce-4198-a831-b8ff3f11fc56 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:16.976945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80ceee-ebce-4198-a831-b8ff3f11fc56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:16.977244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80ceee-ebce-4198-a831-b8ff3f11fc56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:16.978836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80ceee-ebce-4198-a831-b8ff3f11fc56 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:16.992983 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 593/1219] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:17.001009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd15ee-cc57-43fb-9fbf-96a28b8962bd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.018414 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbd15ee-cc57-43fb-9fbf-96a28b8962bd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.029022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.029665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.029940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.030213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.030866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.033023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.095006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918304e8-57ae-4522-882a-577f103b175d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.095205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918304e8-57ae-4522-882a-577f103b175d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.095633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918304e8-57ae-4522-882a-577f103b175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.103796 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 627/1220] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:16 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:17.113357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.129090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-918304e8-57ae-4522-882a-577f103b175d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.130306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-918304e8-57ae-4522-882a-577f103b175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.237701 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 594/1221] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:17.239542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8c728-9ba3-4b27-9047-40e95b1b1ada admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.248290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38e79f-67ca-4406-87e5-deb217011914 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.248657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38e79f-67ca-4406-87e5-deb217011914 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.248871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38e79f-67ca-4406-87e5-deb217011914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.254292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168c7115-ab47-4fb9-b954-294ecd3ac2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.254929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c7115-ab47-4fb9-b954-294ecd3ac2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.255204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c7115-ab47-4fb9-b954-294ecd3ac2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.255474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168c7115-ab47-4fb9-b954-294ecd3ac2f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.256128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c7115-ab47-4fb9-b954-294ecd3ac2f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.257348 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-168c7115-ab47-4fb9-b954-294ecd3ac2f0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:55:17.257348 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:55:17.257348 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:55:17.258695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168c7115-ab47-4fb9-b954-294ecd3ac2f0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.277168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2c38e79f-67ca-4406-87e5-deb217011914 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.278506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2c38e79f-67ca-4406-87e5-deb217011914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.293485 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 628/1222] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/users => generated 292 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:55:17.295702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216299-1a39-49ea-874a-19b615c35d9c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.296219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216299-1a39-49ea-874a-19b615c35d9c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.296795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab216299-1a39-49ea-874a-19b615c35d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.300541 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 595/1223] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:17.301396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38e79f-67ca-4406-87e5-deb217011914 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.323740 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ab216299-1a39-49ea-874a-19b615c35d9c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.325901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ab216299-1a39-49ea-874a-19b615c35d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.330980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38e79f-67ca-4406-87e5-deb217011914 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.337794 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 629/1224] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.338990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216299-1a39-49ea-874a-19b615c35d9c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.346859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb3dfc-1c9a-48c0-a49b-016cc3976e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.347427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb3dfc-1c9a-48c0-a49b-016cc3976e9d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.347701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb3dfc-1c9a-48c0-a49b-016cc3976e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.347971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb3dfc-1c9a-48c0-a49b-016cc3976e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.348657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb3dfc-1c9a-48c0-a49b-016cc3976e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.350254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb3dfc-1c9a-48c0-a49b-016cc3976e9d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.355988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab216299-1a39-49ea-874a-19b615c35d9c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.361472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.363063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.363568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.364068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.365000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:17.365762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.367004 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:55:17.367004 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:55:17.367004 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:55:17.368431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b40b3-7bcc-4094-bb5f-3c2f3dc5773b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.374466 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 596/1225] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:17.375747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699d3e-a880-401f-a769-7745f31a6483 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.376030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699d3e-a880-401f-a769-7745f31a6483 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.376316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37699d3e-a880-401f-a769-7745f31a6483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.384082 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 630/1226] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:17.385957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8da6-771f-4b87-8a8d-a442b50aed66 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.386470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8da6-771f-4b87-8a8d-a442b50aed66 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.386983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8d8da6-771f-4b87-8a8d-a442b50aed66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.413689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6a8d8da6-771f-4b87-8a8d-a442b50aed66 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.414904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6a8d8da6-771f-4b87-8a8d-a442b50aed66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.417668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-37699d3e-a880-401f-a769-7745f31a6483 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.419424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-37699d3e-a880-401f-a769-7745f31a6483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.423485 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 631/1227] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.428166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d8da6-771f-4b87-8a8d-a442b50aed66 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.430282 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 597/1228] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.431910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17095b-45ed-4c92-b766-afedb490b06e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.432454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17095b-45ed-4c92-b766-afedb490b06e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.432960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17095b-45ed-4c92-b766-afedb490b06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.447861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8d8da6-771f-4b87-8a8d-a442b50aed66 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.453615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.454239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.454525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.454805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.456068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.458508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.462362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0e17095b-45ed-4c92-b766-afedb490b06e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.463827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e17095b-45ed-4c92-b766-afedb490b06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.469977 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 632/1229] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:17.471149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.474450 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 598/1230] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.475956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312cce0-9279-4631-892d-956bb7724494 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.476608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312cce0-9279-4631-892d-956bb7724494 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.477563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312cce0-9279-4631-892d-956bb7724494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.483229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b8df3-a7d4-49e5-a32f-5f078cc9a8d2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.488125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfca2c0-96ab-4b43-85e0-b1b7c38a2f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.488779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca2c0-96ab-4b43-85e0-b1b7c38a2f47 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.489063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca2c0-96ab-4b43-85e0-b1b7c38a2f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.489338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca2c0-96ab-4b43-85e0-b1b7c38a2f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.489971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfca2c0-96ab-4b43-85e0-b1b7c38a2f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.491933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfca2c0-96ab-4b43-85e0-b1b7c38a2f47 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.501743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2312cce0-9279-4631-892d-956bb7724494 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.503149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2312cce0-9279-4631-892d-956bb7724494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.514430 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 599/1231] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.521216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312cce0-9279-4631-892d-956bb7724494 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.527748 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 633/1232] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:17.529349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bc210-a329-4ed1-a7ed-f10c0c681357 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.529748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bc210-a329-4ed1-a7ed-f10c0c681357 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.529919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0bc210-a329-4ed1-a7ed-f10c0c681357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.535368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312cce0-9279-4631-892d-956bb7724494 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.550375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ec0bc210-a329-4ed1-a7ed-f10c0c681357 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.551760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ec0bc210-a329-4ed1-a7ed-f10c0c681357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.591272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9887e12-217c-4c62-90a7-c2574bfa7be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.591882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887e12-217c-4c62-90a7-c2574bfa7be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.592163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887e12-217c-4c62-90a7-c2574bfa7be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.592593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9887e12-217c-4c62-90a7-c2574bfa7be3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.593241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887e12-217c-4c62-90a7-c2574bfa7be3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:17.593544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887e12-217c-4c62-90a7-c2574bfa7be3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.595127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9887e12-217c-4c62-90a7-c2574bfa7be3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.598576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:17.607583 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 600/1233] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:17.608910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86668c-9732-4941-b8d9-cb20e830b470 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.609126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86668c-9732-4941-b8d9-cb20e830b470 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.609372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da86668c-9732-4941-b8d9-cb20e830b470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.616014 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 634/1234] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.616014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bc210-a329-4ed1-a7ed-f10c0c681357 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.624910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0bc210-a329-4ed1-a7ed-f10c0c681357 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.629726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71777ffc-2113-4751-8048-50feb197b788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.630370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71777ffc-2113-4751-8048-50feb197b788 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.630646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71777ffc-2113-4751-8048-50feb197b788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.630914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71777ffc-2113-4751-8048-50feb197b788 admin admin] PATH_INFO: `/v3/projects/79e9ffd0be894aca9b7add142f4a54e2/users/bd7fc80eac5d4b2e85e6339acf066b9e/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.632632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-da86668c-9732-4941-b8d9-cb20e830b470 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.633877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da86668c-9732-4941-b8d9-cb20e830b470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.641760 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 601/1235] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.643611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c765d-2367-4473-8bbf-55dad867312f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.643891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c765d-2367-4473-8bbf-55dad867312f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.644164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c765d-2367-4473-8bbf-55dad867312f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.651342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71777ffc-2113-4751-8048-50feb197b788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e9ffd0be894aca9b7add142f4a54e2, user_id=bd7fc80eac5d4b2e85e6339acf066b9e, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.654104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71777ffc-2113-4751-8048-50feb197b788 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.666969 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 635/1236] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:17 2022] PUT /identity/v3/projects/79e9ffd0be894aca9b7add142f4a54e2/users/bd7fc80eac5d4b2e85e6339acf066b9e/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:17.667707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5f0c765d-2367-4473-8bbf-55dad867312f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.668332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e09a2f-9145-4385-b8db-5dd4c817785c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.668750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e09a2f-9145-4385-b8db-5dd4c817785c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.668963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f0c765d-2367-4473-8bbf-55dad867312f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.669073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e09a2f-9145-4385-b8db-5dd4c817785c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.688243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-61e09a2f-9145-4385-b8db-5dd4c817785c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.689454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-61e09a2f-9145-4385-b8db-5dd4c817785c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.696335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:17.729901 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 602/1237] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.731257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10195c5-c688-4bc4-8dcb-c5c61ff51134 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.731541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10195c5-c688-4bc4-8dcb-c5c61ff51134 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.731806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10195c5-c688-4bc4-8dcb-c5c61ff51134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.749895 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 636/1238] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.750869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e09a2f-9145-4385-b8db-5dd4c817785c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.761726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c10195c5-c688-4bc4-8dcb-c5c61ff51134 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.762939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c10195c5-c688-4bc4-8dcb-c5c61ff51134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.763058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e09a2f-9145-4385-b8db-5dd4c817785c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.767991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.768656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.768939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.769213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.769842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.774869 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 603/1239] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.775020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.775114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10195c5-c688-4bc4-8dcb-c5c61ff51134 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.781969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10195c5-c688-4bc4-8dcb-c5c61ff51134 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.789084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd556d71-07f0-4522-a2fc-2cf1019724cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.789938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556d71-07f0-4522-a2fc-2cf1019724cd admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.790386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556d71-07f0-4522-a2fc-2cf1019724cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.790809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556d71-07f0-4522-a2fc-2cf1019724cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.791593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556d71-07f0-4522-a2fc-2cf1019724cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.792821 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-bd556d71-07f0-4522-a2fc-2cf1019724cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:55:17.792821 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:55:17.792821 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:55:17.794544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd556d71-07f0-4522-a2fc-2cf1019724cd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.797167 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 637/1240] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:17.798228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.814725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf683e6c-3e46-454e-a55e-ed728af95a58 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.820089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2ed21-b675-4465-bd65-1c1d0ffc2dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.820748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2ed21-b675-4465-bd65-1c1d0ffc2dde admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.821026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2ed21-b675-4465-bd65-1c1d0ffc2dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.821297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2ed21-b675-4465-bd65-1c1d0ffc2dde admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.821935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2ed21-b675-4465-bd65-1c1d0ffc2dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:17.822234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2ed21-b675-4465-bd65-1c1d0ffc2dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.823824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2ed21-b675-4465-bd65-1c1d0ffc2dde admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.829006 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 604/1241] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:17.829006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac691-a859-49ab-8449-cf8c8415e9b5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.829006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac691-a859-49ab-8449-cf8c8415e9b5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.829006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac691-a859-49ab-8449-cf8c8415e9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.841314 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 638/1242] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:17.842556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233b3e-d46f-4531-8faa-fdf760fbba97 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.842840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233b3e-d46f-4531-8faa-fdf760fbba97 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.843112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17233b3e-d46f-4531-8faa-fdf760fbba97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.852854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-135ac691-a859-49ab-8449-cf8c8415e9b5 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.854003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-135ac691-a859-49ab-8449-cf8c8415e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.871027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-17233b3e-d46f-4531-8faa-fdf760fbba97 None None] MFA Rules not processed for user `bd7fc80eac5d4b2e85e6339acf066b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.872199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-17233b3e-d46f-4531-8faa-fdf760fbba97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.914076 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 605/1243] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.915120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ac691-a859-49ab-8449-cf8c8415e9b5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.924719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ac691-a859-49ab-8449-cf8c8415e9b5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.929181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.929781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.930034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.930300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.930942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:17.931237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.932842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.939071 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 639/1244] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.940458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b645d-a14b-4a2d-bd8e-296c844d12c3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.940743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b645d-a14b-4a2d-bd8e-296c844d12c3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.941020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071b645d-a14b-4a2d-bd8e-296c844d12c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.955206 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 606/1245] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:17.955206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.962604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-071b645d-a14b-4a2d-bd8e-296c844d12c3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:17.963865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-071b645d-a14b-4a2d-bd8e-296c844d12c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:17.965568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8988e-e019-4ea7-a534-0564db878712 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.970373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.971591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.971943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.972181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.972916 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 640/1246] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:17.973708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b645d-a14b-4a2d-bd8e-296c844d12c3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:17.976527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:17.976863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.978451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.984146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071b645d-a14b-4a2d-bd8e-296c844d12c3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:17.991741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:17.992079 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 607/1247] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:17.992460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:17.992790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:17.993006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:17.993810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:17.995413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:17.997186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.005952 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 641/1248] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.006775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.007721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8029bd3e-d2c0-4895-9ab7-6ce099bac725 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.012705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.013305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.013581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.013857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.014481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.016167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3f3e8-59ca-48fa-8883-ae7855c3602e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.016389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.020983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.021553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.021831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.022100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.022734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.024626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.035404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:18.039251 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 608/1249] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:17 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.040557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.052377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9a688-2d03-4868-adad-d4afa0268c45 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.057030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46687aba-f6e2-4ee2-9282-18355c6a894c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.057627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687aba-f6e2-4ee2-9282-18355c6a894c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.061064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687aba-f6e2-4ee2-9282-18355c6a894c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.061196 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 642/1250] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.061390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46687aba-f6e2-4ee2-9282-18355c6a894c admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.061509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.061623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46687aba-f6e2-4ee2-9282-18355c6a894c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:18.061623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46687aba-f6e2-4ee2-9282-18355c6a894c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.061623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46687aba-f6e2-4ee2-9282-18355c6a894c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.073703 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 609/1251] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:18.074907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565021c-4472-4bb0-bb32-ba1b8717c10c None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.075178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565021c-4472-4bb0-bb32-ba1b8717c10c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.075451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565021c-4472-4bb0-bb32-ba1b8717c10c None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.077125 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 610/1252] 10.222.0.39 () {54 vars in 961 bytes} [Sun Oct 23 00:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:55:18.077830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565021c-4472-4bb0-bb32-ba1b8717c10c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.121578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964714ad-eeae-411a-9456-15d8c73ede63 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.126243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.126843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.127118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.127896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.128674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:18.128793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.130679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.146237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565021c-4472-4bb0-bb32-ba1b8717c10c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.151242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.151889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.152173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.152475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.153111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:18.153410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.155010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.157604 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 643/1253] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.158434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.174037 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 611/1254] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:18.174037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.178702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e671d8-eab4-4f87-84ad-e992316795f4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.227296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2e75c-a029-44c6-9b50-9b3afa2221ea admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.231910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.232551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.232835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.233105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.233730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.235612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.235840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.236449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.237238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.237509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.238129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.239682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.249584 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 612/1255] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.250190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.260812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846b31c-2c06-42bd-ab56-7351ed48a07f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.265324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.268995 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 644/1256] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/projects => generated 379 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.269150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.269150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.269150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.269367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.269448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.269448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.279912 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 613/1257] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.280740 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.290183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16ed236-bb4f-4bc7-a6af-73cf102d6ba9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.331066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9854286-f2ae-444e-a17d-7d0c7e78b07c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.335868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4018ad3d-f073-416a-98f9-5ad91ccefbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.336511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ad3d-f073-416a-98f9-5ad91ccefbb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.336789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ad3d-f073-416a-98f9-5ad91ccefbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.337060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ad3d-f073-416a-98f9-5ad91ccefbb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.337693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018ad3d-f073-416a-98f9-5ad91ccefbb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.339616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4018ad3d-f073-416a-98f9-5ad91ccefbb4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.346805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.347442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.347566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:18.347693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.348265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.348621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.350578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.363614 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 645/1258] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/users => generated 297 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.365184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518ee7-f3df-4751-8009-6690cec8963f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.365647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518ee7-f3df-4751-8009-6690cec8963f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.366080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e518ee7-f3df-4751-8009-6690cec8963f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.383527 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 614/1259] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.383527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.409613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3e518ee7-f3df-4751-8009-6690cec8963f None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:18.410828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e518ee7-f3df-4751-8009-6690cec8963f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:18.433567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:18.446103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c894092-8a57-4413-9ade-4eb5cb293b2c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.450616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.451216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.451493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.451763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.452408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.454366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.475519 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 615/1260] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.476413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.478365 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 646/1261] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:18.479095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e518ee7-f3df-4751-8009-6690cec8963f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.489000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcb35ea-be5f-4cab-830b-f9624e10f368 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.490236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e518ee7-f3df-4751-8009-6690cec8963f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.493551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.494139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.494416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.494675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.494789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.495287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.495396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.495692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.495971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] PATH_INFO: `/v3/projects/11730d9eec8546c5aa486c1a4a283bf8/users/992874ab1b9c4fce90b46e659910d512/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.497273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.511338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=11730d9eec8546c5aa486c1a4a283bf8, user_id=992874ab1b9c4fce90b46e659910d512, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.513996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.525513 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 616/1262] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.526670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 647/1263] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:18 2022] PUT /identity/v3/projects/11730d9eec8546c5aa486c1a4a283bf8/users/992874ab1b9c4fce90b46e659910d512/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:18.526960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.527446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.587935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d75e2-5df0-4f09-a325-31f1b9eb3606 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.588266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6f1c66-2cf8-492c-b837-4416687289ff admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.592708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.592845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.593316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.593427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.593593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.593690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.593862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.593973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.594483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.594586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.596640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.596934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.606430 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 648/1264] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.607286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.618451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37859634-cd2f-4a77-9a76-f0736bcf1ba4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.624889 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 617/1265] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.624889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.625154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.625154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.625154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.625154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.625506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.626977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.632323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679816f1-1597-4eb8-bf03-3f58bfffe90e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.636904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.637039 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 649/1266] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.637438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.637717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.637841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.637989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.638616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.640569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.647343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516021-113c-4bbc-ae8f-7a5c45ef8349 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.651963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.652620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.652947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.653290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.654023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.655730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.662435 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 618/1267] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/users => generated 293 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.663273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.670496 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 650/1268] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.671695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.682071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d865ee-e2d7-4dd7-aa3e-f1df35f46ba9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.686585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b585c854-17b6-43fd-975f-1fc7c31e23f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.687156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c854-17b6-43fd-975f-1fc7c31e23f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.687431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c854-17b6-43fd-975f-1fc7c31e23f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.687699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b585c854-17b6-43fd-975f-1fc7c31e23f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.688322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585c854-17b6-43fd-975f-1fc7c31e23f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.690242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b585c854-17b6-43fd-975f-1fc7c31e23f6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.706377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:18.712193 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 651/1269] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:18.713636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82ccf-1fce-41de-9fe1-c9ff815fd912 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.714053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82ccf-1fce-41de-9fe1-c9ff815fd912 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.714542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd82ccf-1fce-41de-9fe1-c9ff815fd912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.733735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2cd82ccf-1fce-41de-9fe1-c9ff815fd912 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:18.734928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2cd82ccf-1fce-41de-9fe1-c9ff815fd912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:18.739226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cff2e8e-be1f-4fee-a68e-14d646edd796 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.743074 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 652/1270] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:18.743814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd82ccf-1fce-41de-9fe1-c9ff815fd912 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.748638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.749255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.749529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.749798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.753976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd82ccf-1fce-41de-9fe1-c9ff815fd912 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.758553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.759194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.759474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.759744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] PATH_INFO: `/v3/projects/7103bcb49d5c4e8b958fba3b1309a881/users/11b0f829f5654010b2bd1a76888d7ebf/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.774252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7103bcb49d5c4e8b958fba3b1309a881, user_id=11b0f829f5654010b2bd1a76888d7ebf, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.776653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:18.776937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.790573 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 653/1271] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:18 2022] PUT /identity/v3/projects/7103bcb49d5c4e8b958fba3b1309a881/users/11b0f829f5654010b2bd1a76888d7ebf/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:18.794319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.809970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.811767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.821393 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 619/1272] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:18.822272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.833310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9a892-cd5d-4392-8c03-b7cea0204a5f service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.857235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be18c-2105-46c2-b36a-7c1d00bcd22c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.862455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.863164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.863451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.863725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.864424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.866129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.877250 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 654/1273] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.878055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.888494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34cdbc-3411-4c43-80b9-149afe065dc1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.894538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.895125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.895411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.895750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.896313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.897923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.901649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.902365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.902771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.903239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.904000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.905745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.907848 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 655/1274] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.909218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.920372 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 620/1275] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.920372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.920643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad79a0b-985c-45a2-817c-1178a0dfe3d8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.924515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.925085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.925372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.925629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.926245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.927834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.930788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f67c-3f01-4eb6-b5f9-9f8aa2baef8c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:18.935766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8728aa-f0d5-45f0-9e97-9f5bc7276cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:18.936484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8728aa-f0d5-45f0-9e97-9f5bc7276cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.936864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8728aa-f0d5-45f0-9e97-9f5bc7276cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.937277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8728aa-f0d5-45f0-9e97-9f5bc7276cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.937994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8728aa-f0d5-45f0-9e97-9f5bc7276cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:18.942289 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 656/1276] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.942289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.942874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8728aa-f0d5-45f0-9e97-9f5bc7276cc3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:18.953909 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 621/1277] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:18.953909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eee4f-a351-483b-b7b8-58704037308f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:18.953909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eee4f-a351-483b-b7b8-58704037308f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:18.953909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6eee4f-a351-483b-b7b8-58704037308f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:18.973664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0c6eee4f-a351-483b-b7b8-58704037308f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:18.975032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0c6eee4f-a351-483b-b7b8-58704037308f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:18.984975 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 622/1278] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:18.985174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6eee4f-a351-483b-b7b8-58704037308f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:18.995767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6eee4f-a351-483b-b7b8-58704037308f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.000888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.001611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.001985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.002523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.003529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.005286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.015669 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 623/1279] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:19.016682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.024477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae358ea-bef3-4dc9-9f68-f0c7b8e17a8c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.027984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.029012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.029422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.029700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.035514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb249a1-8909-4797-8c49-1d77a42c2352 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.040521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.040648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.041154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.041532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.041914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.042179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.042742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.044463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.053118 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 657/1280] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:19.054144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.063423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e59c7f-6f0d-4e22-9d2a-3671134d989e service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.067970 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 624/1281] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:19.069012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.069147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.069894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.070276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.070786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] PATH_INFO: `/v3/projects/ebac7696faf14dfea4957e97b589029d/users/28563be0db564f8ebcf765a78970ce8f/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.087448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eed903-41ae-4ebf-998f-a8dbd44907b9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.088123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebac7696faf14dfea4957e97b589029d, user_id=28563be0db564f8ebcf765a78970ce8f, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.090970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.093065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.093671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.093948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.094780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] PATH_INFO: `/v3/projects/e6021466a9b34c4fa7428ef782da2be2/users/5a0eae8d97a743f292b9bc76cc4558e8/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.105658 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 658/1282] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:19 2022] PUT /identity/v3/projects/ebac7696faf14dfea4957e97b589029d/users/28563be0db564f8ebcf765a78970ce8f/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:19.107013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.113912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6021466a9b34c4fa7428ef782da2be2, user_id=5a0eae8d97a743f292b9bc76cc4558e8, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.118017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.133824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:19.135776 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 625/1283] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:19 2022] PUT /identity/v3/projects/e6021466a9b34c4fa7428ef782da2be2/users/5a0eae8d97a743f292b9bc76cc4558e8/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:19.136708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.180531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df4e64-cc0b-4494-aba4-f85130109907 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.205864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a76d2d-27be-4ab4-ac51-794c3197da4a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.210668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.211533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.212036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.212445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.227786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:19.260728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.261566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.262032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.262944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.263661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.266058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.290367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.292164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.295793 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 659/1284] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:19.296805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.308211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae682c-afc5-4bf1-8713-565482103aab admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.309167 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 626/1285] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:19.309999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.313957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39392869-d534-480a-9f62-91e403588ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.315771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392869-d534-480a-9f62-91e403588ef8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.316155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392869-d534-480a-9f62-91e403588ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.317311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39392869-d534-480a-9f62-91e403588ef8 admin admin] PATH_INFO: `/v3/projects/c7e74b2109214d21bb658dc423b38505/users/902cd162a23540d7b458a9f8b432b6c4/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.321473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30730ff-36d8-4f42-ad0f-bcff5411a995 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.326304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f865cac-98f2-4c0d-b6c6-1cc91910ce09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.327031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f865cac-98f2-4c0d-b6c6-1cc91910ce09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.327314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f865cac-98f2-4c0d-b6c6-1cc91910ce09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.327589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f865cac-98f2-4c0d-b6c6-1cc91910ce09 admin admin] PATH_INFO: `/v3/projects/f639829059494fd49c12d49d52134692/users/959991a17c69459dbd665c2ae20095ae/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.338627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39392869-d534-480a-9f62-91e403588ef8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7e74b2109214d21bb658dc423b38505, user_id=902cd162a23540d7b458a9f8b432b6c4, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.344492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39392869-d534-480a-9f62-91e403588ef8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.349903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f865cac-98f2-4c0d-b6c6-1cc91910ce09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f639829059494fd49c12d49d52134692, user_id=959991a17c69459dbd665c2ae20095ae, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.352905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f865cac-98f2-4c0d-b6c6-1cc91910ce09 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.362986 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 660/1286] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:19 2022] PUT /identity/v3/projects/c7e74b2109214d21bb658dc423b38505/users/902cd162a23540d7b458a9f8b432b6c4/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:19.364357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819ef85-a759-4f75-8070-91d70982465c None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.364639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819ef85-a759-4f75-8070-91d70982465c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.364911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819ef85-a759-4f75-8070-91d70982465c None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.366880 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 661/1287] 10.222.0.39 () {54 vars in 912 bytes} [Sun Oct 23 00:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:55:19.368378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e2b3b-718d-45de-ba34-a68a9ed6bc69 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.368727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e2b3b-718d-45de-ba34-a68a9ed6bc69 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.369062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e2b3b-718d-45de-ba34-a68a9ed6bc69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.373171 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 627/1288] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:19 2022] PUT /identity/v3/projects/f639829059494fd49c12d49d52134692/users/959991a17c69459dbd665c2ae20095ae/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:19.374499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e11f4-70c6-45d1-a033-5dee4f54f259 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.374804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e11f4-70c6-45d1-a033-5dee4f54f259 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.375083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449e11f4-70c6-45d1-a033-5dee4f54f259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.396974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-951e2b3b-718d-45de-ba34-a68a9ed6bc69 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:19.399324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-951e2b3b-718d-45de-ba34-a68a9ed6bc69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:19.406721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-449e11f4-70c6-45d1-a033-5dee4f54f259 None None] MFA Rules not processed for user `5a0eae8d97a743f292b9bc76cc4558e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:19.408233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-449e11f4-70c6-45d1-a033-5dee4f54f259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:19.480481 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 662/1289] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:19.481520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e2b3b-718d-45de-ba34-a68a9ed6bc69 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.492823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e2b3b-718d-45de-ba34-a68a9ed6bc69 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.498141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1022467d-c8f2-48fa-9e12-f055657c429f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.498949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022467d-c8f2-48fa-9e12-f055657c429f admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.499322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022467d-c8f2-48fa-9e12-f055657c429f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.499687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1022467d-c8f2-48fa-9e12-f055657c429f admin admin] PATH_INFO: `/v3/projects/30b63353122946aa939b666951872ab0/users/169934ffb3364fabb08181fa5fa3a2df/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.509837 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 628/1290] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:19.511395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d1b-c29d-46a3-baea-330b2720d7f3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.511861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d1b-c29d-46a3-baea-330b2720d7f3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.512317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab0d1b-c29d-46a3-baea-330b2720d7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.526305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022467d-c8f2-48fa-9e12-f055657c429f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30b63353122946aa939b666951872ab0, user_id=169934ffb3364fabb08181fa5fa3a2df, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.597821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1022467d-c8f2-48fa-9e12-f055657c429f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.597821 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 663/1291] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:19 2022] PUT /identity/v3/projects/30b63353122946aa939b666951872ab0/users/169934ffb3364fabb08181fa5fa3a2df/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:19.597821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.597821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.597821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.598351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d8ab0d1b-c29d-46a3-baea-330b2720d7f3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:19.598351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d8ab0d1b-c29d-46a3-baea-330b2720d7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:19.613553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:19.615012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:19.640128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:19.707187 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 629/1292] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:19.708609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ba1a3-f8f6-43de-9f67-77099d1e8c85 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.710987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ba1a3-f8f6-43de-9f67-77099d1e8c85 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.712115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0ba1a3-f8f6-43de-9f67-77099d1e8c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.748785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5d0ba1a3-f8f6-43de-9f67-77099d1e8c85 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:19.748785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5d0ba1a3-f8f6-43de-9f67-77099d1e8c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:19.756738 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 664/1293] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:19.757898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.760472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:19.769578 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 630/1294] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:19.771255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62df687-4886-42fa-b10c-ce8805e98ee1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.771255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62df687-4886-42fa-b10c-ce8805e98ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.771839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62df687-4886-42fa-b10c-ce8805e98ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.802405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:19.819649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f62df687-4886-42fa-b10c-ce8805e98ee1 None None] MFA Rules not processed for user `169934ffb3364fabb08181fa5fa3a2df`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:19.821001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f62df687-4886-42fa-b10c-ce8805e98ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:19.897980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a44b5-6614-4759-976e-0a1464cc3cde None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.903806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.904533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.904810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.905217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.917888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.921247 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 631/1295] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:19.921861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.923136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457271db-ec70-4c90-be2f-30bae2a7378d None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.923713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457271db-ec70-4c90-be2f-30bae2a7378d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.924096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457271db-ec70-4c90-be2f-30bae2a7378d None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.925666 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 632/1296] 10.222.0.39 () {54 vars in 961 bytes} [Sun Oct 23 00:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:55:19.927347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457271db-ec70-4c90-be2f-30bae2a7378d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.942139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457271db-ec70-4c90-be2f-30bae2a7378d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:19.947144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:19.948235 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 665/1297] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:19.949381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:19.949519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:19.950106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:19.951572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:19.973474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:19.977677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:19.995065 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 633/1298] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:19.996145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:20.020366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3d717c-c13b-4a1e-834c-b02ae545d5e4 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:20.029069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638b162-da02-4241-ab41-7e6da03fefb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:20.030225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638b162-da02-4241-ab41-7e6da03fefb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.030763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638b162-da02-4241-ab41-7e6da03fefb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.031572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638b162-da02-4241-ab41-7e6da03fefb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.048173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638b162-da02-4241-ab41-7e6da03fefb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:20.051792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638b162-da02-4241-ab41-7e6da03fefb5 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:20.071278 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 634/1299] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:20.073496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79718d-7d3c-415b-a1f0-5c17e85d6f07 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.073936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79718d-7d3c-415b-a1f0-5c17e85d6f07 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.074311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79718d-7d3c-415b-a1f0-5c17e85d6f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.106474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ef79718d-7d3c-415b-a1f0-5c17e85d6f07 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:20.110378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ef79718d-7d3c-415b-a1f0-5c17e85d6f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:20.125479 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 635/1300] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Oct 23 00:55:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:20.131596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157d3e36-ed1e-4e79-bd5d-1af5892b7691 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:20.137698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79718d-7d3c-415b-a1f0-5c17e85d6f07 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:20.140447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:20.141576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.142159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.142689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.150891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79718d-7d3c-415b-a1f0-5c17e85d6f07 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:20.159428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:20.160923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8d4a8-5c66-4db0-8a77-690d31937e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:20.161972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d4a8-5c66-4db0-8a77-690d31937e51 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.162255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d4a8-5c66-4db0-8a77-690d31937e51 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.162527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8d4a8-5c66-4db0-8a77-690d31937e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.172339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:20.187250 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 666/1301] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:20.298866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8d4a8-5c66-4db0-8a77-690d31937e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:20.302403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8d4a8-5c66-4db0-8a77-690d31937e51 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:20.335488 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 636/1302] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:55:20.606343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:20.625605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2de73-541d-46c5-958f-76518c770446 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:20.634984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f28588-9ce9-45fa-b746-3852f5ad26a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:20.635645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f28588-9ce9-45fa-b746-3852f5ad26a4 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.635959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f28588-9ce9-45fa-b746-3852f5ad26a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.636699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f28588-9ce9-45fa-b746-3852f5ad26a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.647429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f28588-9ce9-45fa-b746-3852f5ad26a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:20.651191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f28588-9ce9-45fa-b746-3852f5ad26a4 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:20.663262 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 667/1303] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:20.741856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e527b5-cdab-4841-8367-96f7da709dbc None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.742538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e527b5-cdab-4841-8367-96f7da709dbc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.743036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e527b5-cdab-4841-8367-96f7da709dbc None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.745001 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 637/1304] 10.222.0.39 () {54 vars in 912 bytes} [Sun Oct 23 00:55:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:55:20.757016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39723e-d498-4ef6-a177-5dc6b3615fd3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.757379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39723e-d498-4ef6-a177-5dc6b3615fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.757664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39723e-d498-4ef6-a177-5dc6b3615fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.800028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fa39723e-d498-4ef6-a177-5dc6b3615fd3 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:20.801441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fa39723e-d498-4ef6-a177-5dc6b3615fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:20.838082 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 668/1305] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:55:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:20.853689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e527b5-cdab-4841-8367-96f7da709dbc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:20.876928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e527b5-cdab-4841-8367-96f7da709dbc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:20.892280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9a5c7c-cca2-471a-9c82-bb85660ff252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:20.894279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a5c7c-cca2-471a-9c82-bb85660ff252 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:20.894570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a5c7c-cca2-471a-9c82-bb85660ff252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:20.894979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9a5c7c-cca2-471a-9c82-bb85660ff252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:20.921058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a5c7c-cca2-471a-9c82-bb85660ff252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:20.922795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9a5c7c-cca2-471a-9c82-bb85660ff252 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:20.949781 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 638/1306] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:21.519143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39723e-d498-4ef6-a177-5dc6b3615fd3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:21.532083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39723e-d498-4ef6-a177-5dc6b3615fd3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:21.540246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:21.549937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:21.550302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:21.550874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] PATH_INFO: `/v3/users/5a0eae8d97a743f292b9bc76cc4558e8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:21.554825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0eae8d97a743f292b9bc76cc4558e8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:21.559435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:21.639583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:21.678238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0eae8d97a743f292b9bc76cc4558e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:21.694447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0eae8d97a743f292b9bc76cc4558e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:21.716424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0eae8d97a743f292b9bc76cc4558e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:21.746475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0eae8d97a743f292b9bc76cc4558e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:21.748786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0eae8d97a743f292b9bc76cc4558e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:21.765857 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 669/1307] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:21 2022] DELETE /identity/v3/users/5a0eae8d97a743f292b9bc76cc4558e8 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:21.770314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:21.770666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:21.771012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:21.813594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:21.815396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:21.913918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:21.936151 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 639/1308] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:21.964973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:21.966581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:22.091096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015056bc-a42d-4c1b-ab64-7e82684c676a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:22.100006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2176bee0-3dfc-4036-9957-8de2ca9d57c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:22.101066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bee0-3dfc-4036-9957-8de2ca9d57c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:22.101758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bee0-3dfc-4036-9957-8de2ca9d57c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:22.103265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176bee0-3dfc-4036-9957-8de2ca9d57c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:22.133195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176bee0-3dfc-4036-9957-8de2ca9d57c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:22.138545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2176bee0-3dfc-4036-9957-8de2ca9d57c0 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:22.159275 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 670/1309] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:23.208725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:23.216754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:23.250966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbca0ef-4d50-4557-b9a5-e155ed3f8a24 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:23.257858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:23.259233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:23.259678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:23.260122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] PATH_INFO: `/v3/projects/e6021466a9b34c4fa7428ef782da2be2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:23.261734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6021466a9b34c4fa7428ef782da2be2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:23.264681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:23.411533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6021466a9b34c4fa7428ef782da2be2', 'request_id': 'req-24675b43-740e-4b56-b87f-b71a4692b4f2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:23.431167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6021466a9b34c4fa7428ef782da2be2', 'request_id': 'req-24675b43-740e-4b56-b87f-b71a4692b4f2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:23.445548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6021466a9b34c4fa7428ef782da2be2', 'request_id': 'req-24675b43-740e-4b56-b87f-b71a4692b4f2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:23.465357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] The token cache is being invalidate because project e6021466a9b34c4fa7428ef782da2be2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:23.467261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:23.469733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-24675b43-740e-4b56-b87f-b71a4692b4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:23.474963 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 640/1310] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:23 2022] DELETE /identity/v3/projects/e6021466a9b34c4fa7428ef782da2be2 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:23.479841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e4394-a3f7-45aa-956a-6718513b33c8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:23.480590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e4394-a3f7-45aa-956a-6718513b33c8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:23.480590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e4394-a3f7-45aa-956a-6718513b33c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:23.527547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e12e4394-a3f7-45aa-956a-6718513b33c8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:23.530135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e12e4394-a3f7-45aa-956a-6718513b33c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:23.707645 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 671/1311] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:23.715215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869da21-b1ce-4348-93ab-6efbaeb57cd5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:23.716563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869da21-b1ce-4348-93ab-6efbaeb57cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:23.716855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869da21-b1ce-4348-93ab-6efbaeb57cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:23.806845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a869da21-b1ce-4348-93ab-6efbaeb57cd5 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:23.808891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a869da21-b1ce-4348-93ab-6efbaeb57cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:23.838549 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 641/1312] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:23.842880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e4394-a3f7-45aa-956a-6718513b33c8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:23.859178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e4394-a3f7-45aa-956a-6718513b33c8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:23.879953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:23.880614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:23.880898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:23.881174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:23.887322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:23.890061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:23.905883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:23.950245 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 672/1313] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:23.951570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869da21-b1ce-4348-93ab-6efbaeb57cd5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:23.966046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869da21-b1ce-4348-93ab-6efbaeb57cd5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:23.997849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:23.999391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:23.999836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:24.000627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:24.002426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:24.006039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:24.082307 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 642/1314] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:23 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:24.086769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:24.101839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937da52c-bb69-4bec-a4de-ae59f77d47e6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:24.169093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:24.215292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:24.215292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:24.218861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:24.219152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:24.220001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:24.224965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:24.264866 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 673/1315] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:24 2022] POST /identity/v3/users => generated 303 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:24.269638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:24.292214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e18098-a39b-4c5e-bce7-07dd5b73a1ad admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:24.298154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:24.300337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:24.300864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:24.301352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:24.302231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:24.304051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:24.324168 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 643/1316] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:24.328343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:24.339855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d31406-a332-4fe0-bdf0-177f9ed03269 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:24.346951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff0efd-d76c-4164-87ac-0ee8fc7d317d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:24.348066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff0efd-d76c-4164-87ac-0ee8fc7d317d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:24.348580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff0efd-d76c-4164-87ac-0ee8fc7d317d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:24.349172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff0efd-d76c-4164-87ac-0ee8fc7d317d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:24.350072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff0efd-d76c-4164-87ac-0ee8fc7d317d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:24.352425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff0efd-d76c-4164-87ac-0ee8fc7d317d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:24.372634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:24.373715 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 674/1317] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:24 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:55:24.385700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9418f484-5057-4b35-a72e-645900ade89d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:24.391114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:24.394017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:24.394856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:24.395407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] PATH_INFO: `/v3/projects/7bf497a4063c4290b224f38ed5769852/users/a5feea8839c74bfb9eebfdf0171631c2/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:24.421025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bf497a4063c4290b224f38ed5769852, user_id=a5feea8839c74bfb9eebfdf0171631c2, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:24.424922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:24.462711 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 644/1318] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:24 2022] PUT /identity/v3/projects/7bf497a4063c4290b224f38ed5769852/users/a5feea8839c74bfb9eebfdf0171631c2/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:24.464463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9635fc-10de-4285-827c-edf895911bcc None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:24.464758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9635fc-10de-4285-827c-edf895911bcc None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:24.465035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9635fc-10de-4285-827c-edf895911bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:24.534711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-dd9635fc-10de-4285-827c-edf895911bcc None None] MFA Rules not processed for user `a5feea8839c74bfb9eebfdf0171631c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:24.536097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dd9635fc-10de-4285-827c-edf895911bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:24.694279 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 675/1319] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:24 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:24.698955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:24.824433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23c2919-93a3-45b5-9547-94f6dbbcbfa3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:24.834069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:24.834903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:24.835306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:24.835756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:24.836555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:24.839657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:24.863785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:24.882355 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 645/1320] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:24 2022] POST /identity/v3/projects => generated 405 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:24.887193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9635fc-10de-4285-827c-edf895911bcc None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:24.901193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9635fc-10de-4285-827c-edf895911bcc None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:24.987350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:24.990136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:25.041328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:25.042124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.042539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.042920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.048590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:25.055511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:25.093681 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 676/1321] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:24 2022] POST /identity/v3/users => generated 309 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:25.096719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:25.109634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad08e2-99c9-4880-ab58-f76910bb797b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:25.118289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:25.119765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.121781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.122489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.125033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:25.125960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adba4fc-6fa0-4b4c-b49b-242978da068f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:25.135971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:25.137826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:25.138506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.138783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.139050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] PATH_INFO: `/v3/users/169934ffb3364fabb08181fa5fa3a2df` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.143615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=169934ffb3364fabb08181fa5fa3a2df)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:25.144708 dv-cinder-master-3-smb devstack@keystone.service[81371]: WARNING py.warnings [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:55:25.144708 dv-cinder-master-3-smb devstack@keystone.service[81371]: warnings.warn(msg) Oct 23 00:55:25.144708 dv-cinder-master-3-smb devstack@keystone.service[81371]: Oct 23 00:55:25.145944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:25.167971 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 677/1322] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:25.172680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:25.194908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3752edc9-6208-4ed1-a047-15baa91577bc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:25.210906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:25.212110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.212110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.212361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] PATH_INFO: `/v3/projects/80bd8dbd74ed46cfb932b8e4f6e88585/users/8abe39e55ff745acbe0c075af6cc4fc2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.256725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bd8dbd74ed46cfb932b8e4f6e88585, user_id=8abe39e55ff745acbe0c075af6cc4fc2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:25.263381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:25.270541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '169934ffb3364fabb08181fa5fa3a2df'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:25.271446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '169934ffb3364fabb08181fa5fa3a2df'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:25.293345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:25.302424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 678/1323] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:25 2022] PUT /identity/v3/projects/80bd8dbd74ed46cfb932b8e4f6e88585/users/8abe39e55ff745acbe0c075af6cc4fc2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:25.302575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '169934ffb3364fabb08181fa5fa3a2df'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:25.303772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:25.308691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:25.351275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '169934ffb3364fabb08181fa5fa3a2df'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:25.354850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9b4e84ee-0c36-4c0d-b044-bc7b50788d3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '169934ffb3364fabb08181fa5fa3a2df'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:25.390262 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 646/1324] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:24 2022] DELETE /identity/v3/users/169934ffb3364fabb08181fa5fa3a2df => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:25.393958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.395762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.396258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.447722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:25.449782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:25.520635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080f6a3c-bc89-41ca-8916-fb94401d4e1c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:25.615320 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 647/1325] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:25.649199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:25.650952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:25.662856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:25.663473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.663936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.664537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.666077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:25.677418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:25.690112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:25.700131 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 679/1326] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:25 2022] GET /identity/v3/roles => generated 1420 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:25.710154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:25.747904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9c3e36-e29d-48f8-97ee-8f6181e71608 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:25.760675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f04686-a6c1-453e-8b98-e65076b6e095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:25.761513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f04686-a6c1-453e-8b98-e65076b6e095 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.763402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f04686-a6c1-453e-8b98-e65076b6e095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.764482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f04686-a6c1-453e-8b98-e65076b6e095 admin admin] PATH_INFO: `/v3/projects/80bd8dbd74ed46cfb932b8e4f6e88585/users/8abe39e55ff745acbe0c075af6cc4fc2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.770456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f04686-a6c1-453e-8b98-e65076b6e095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80bd8dbd74ed46cfb932b8e4f6e88585, user_id=8abe39e55ff745acbe0c075af6cc4fc2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:25.776987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f04686-a6c1-453e-8b98-e65076b6e095 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:25.794483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24057d74-3950-43e0-894c-29d377f1ce12 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:25.816791 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 680/1327] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:25 2022] PUT /identity/v3/projects/80bd8dbd74ed46cfb932b8e4f6e88585/users/8abe39e55ff745acbe0c075af6cc4fc2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:25.836183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe40b5-712e-48f4-a949-28ae0d5bf629 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.837555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe40b5-712e-48f4-a949-28ae0d5bf629 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.838396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe40b5-712e-48f4-a949-28ae0d5bf629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.888628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:25.889648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:25.890037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:25.890463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:25.945842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3dbe40b5-712e-48f4-a949-28ae0d5bf629 None None] MFA Rules not processed for user `8abe39e55ff745acbe0c075af6cc4fc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:25.947073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3dbe40b5-712e-48f4-a949-28ae0d5bf629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:25.985988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:25.989371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:26.014540 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 648/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:55:26.114322 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 681/1329] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:26.125320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:26.127503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:26.267405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5044eb5-070a-44fa-9898-13ce6ced572e service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:26.281897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704e29d6-c718-4dfb-a0a2-ae73a3697673 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:26.283915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e29d6-c718-4dfb-a0a2-ae73a3697673 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:26.284547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e29d6-c718-4dfb-a0a2-ae73a3697673 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:26.284991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704e29d6-c718-4dfb-a0a2-ae73a3697673 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:26.302576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e29d6-c718-4dfb-a0a2-ae73a3697673 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:26.305321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704e29d6-c718-4dfb-a0a2-ae73a3697673 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:26.322271 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 649/1330] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:26 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:26.629563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe40b5-712e-48f4-a949-28ae0d5bf629 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:26.647394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbe40b5-712e-48f4-a949-28ae0d5bf629 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:26.667335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:26.668065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:26.668920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:26.669212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] PATH_INFO: `/v3/users/a5feea8839c74bfb9eebfdf0171631c2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:26.672354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5feea8839c74bfb9eebfdf0171631c2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:26.674906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:26.718347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:26.880909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5feea8839c74bfb9eebfdf0171631c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:26.906304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5feea8839c74bfb9eebfdf0171631c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:26.935781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5feea8839c74bfb9eebfdf0171631c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:26.961760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5feea8839c74bfb9eebfdf0171631c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:26.962726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5feea8839c74bfb9eebfdf0171631c2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:26.975108 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 682/1331] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:26 2022] DELETE /identity/v3/users/a5feea8839c74bfb9eebfdf0171631c2 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:26.979685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:26.979998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:26.980240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:27.016224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:27.016224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:27.064170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:27.106378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:27.113749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:27.194951 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 650/1332] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:27.235438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:27.238071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:27.284850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fa08c-51b1-4225-808b-ff8e20db1440 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:27.303187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:27.304728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:27.306505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:27.307768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] PATH_INFO: `/v3/users/bd7fc80eac5d4b2e85e6339acf066b9e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:27.312430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd7fc80eac5d4b2e85e6339acf066b9e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:27.319354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:27.357181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8dab8-d996-46ed-b889-9f53f1e9daa1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:27.362194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67026dd-5280-4581-b64b-7fb8fddd2a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:27.363059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67026dd-5280-4581-b64b-7fb8fddd2a9b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:27.363637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67026dd-5280-4581-b64b-7fb8fddd2a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:27.364096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67026dd-5280-4581-b64b-7fb8fddd2a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:27.381671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67026dd-5280-4581-b64b-7fb8fddd2a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:27.385071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67026dd-5280-4581-b64b-7fb8fddd2a9b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:27.404088 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 651/1333] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:55:27.443285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd7fc80eac5d4b2e85e6339acf066b9e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:27.456489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd7fc80eac5d4b2e85e6339acf066b9e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:27.479658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd7fc80eac5d4b2e85e6339acf066b9e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:27.503458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd7fc80eac5d4b2e85e6339acf066b9e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:27.505099 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd7fc80eac5d4b2e85e6339acf066b9e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:27.519268 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 683/1334] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:27 2022] DELETE /identity/v3/users/bd7fc80eac5d4b2e85e6339acf066b9e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:27.524420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:27.525213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:27.525655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:27.566917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:27.568218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:27.723429 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 652/1335] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:27.890880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:27.893415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:27.915991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:28.063040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0d55f6-d7e5-4289-8dd5-96c89082e672 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:28.073895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830ac10-1b81-439a-a482-d40442f2ffc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:28.074727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ac10-1b81-439a-a482-d40442f2ffc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:28.075009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ac10-1b81-439a-a482-d40442f2ffc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:28.075284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ac10-1b81-439a-a482-d40442f2ffc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:28.119034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830ac10-1b81-439a-a482-d40442f2ffc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:28.120177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9830ac10-1b81-439a-a482-d40442f2ffc4 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:28.164859 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 684/1336] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:29.192680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:29.215846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:29.254548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b04c258-c6b9-49e2-8d61-4d1e62b83bda None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:29.261390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:29.262472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:29.262919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:29.263420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] PATH_INFO: `/v3/projects/30b63353122946aa939b666951872ab0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:29.264911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30b63353122946aa939b666951872ab0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:29.267922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:29.416614 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30b63353122946aa939b666951872ab0', 'request_id': 'req-be6c6eb4-fdf7-41ad-bdec-67e506445b62'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:29.442506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30b63353122946aa939b666951872ab0', 'request_id': 'req-be6c6eb4-fdf7-41ad-bdec-67e506445b62'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:29.453771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:29.464646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30b63353122946aa939b666951872ab0', 'request_id': 'req-be6c6eb4-fdf7-41ad-bdec-67e506445b62'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:29.497325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] The token cache is being invalidate because project 30b63353122946aa939b666951872ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:29.497619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:29.512109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-be6c6eb4-fdf7-41ad-bdec-67e506445b62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:29.515412 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 653/1337] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:29 2022] DELETE /identity/v3/projects/30b63353122946aa939b666951872ab0 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:29.520972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65248c-7909-4c68-ba0c-a8cb393356e6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:29.521528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65248c-7909-4c68-ba0c-a8cb393356e6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:29.521528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65248c-7909-4c68-ba0c-a8cb393356e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:29.570938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ad65248c-7909-4c68-ba0c-a8cb393356e6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:29.575577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ad65248c-7909-4c68-ba0c-a8cb393356e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:29.704706 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 685/1338] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:29.713448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52314-0695-423a-b8a0-f587ef61f41f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:29.715387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52314-0695-423a-b8a0-f587ef61f41f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:29.715771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52314-0695-423a-b8a0-f587ef61f41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:29.778037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0cd52314-0695-423a-b8a0-f587ef61f41f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:29.786150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0cd52314-0695-423a-b8a0-f587ef61f41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:29.813927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65248c-7909-4c68-ba0c-a8cb393356e6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:29.818860 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 654/1339] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:29.831990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65248c-7909-4c68-ba0c-a8cb393356e6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:29.848605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117cd152-2b89-4c5c-8a50-acf87c80ea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:29.849729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd152-2b89-4c5c-8a50-acf87c80ea28 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:29.850116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd152-2b89-4c5c-8a50-acf87c80ea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:29.850538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117cd152-2b89-4c5c-8a50-acf87c80ea28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:29.851353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117cd152-2b89-4c5c-8a50-acf87c80ea28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:29.856206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117cd152-2b89-4c5c-8a50-acf87c80ea28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:29.858186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117cd152-2b89-4c5c-8a50-acf87c80ea28 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:29.874563 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 686/1340] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:29.879150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd52314-0695-423a-b8a0-f587ef61f41f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:29.891295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd52314-0695-423a-b8a0-f587ef61f41f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:29.911343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:29.911981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:29.912258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:29.912551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:29.913193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:29.915108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:29.960161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:29.960484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:29.960760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:29.986971 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 655/1341] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:29 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:29.992453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:30.022191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] MFA Rules not processed for user `11b0f829f5654010b2bd1a76888d7ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:30.023813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:30.137624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a95ae-b361-4e59-ac56-68b42748c635 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:30.147132 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 687/1342] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:30.164766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:30.165192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:30.165467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:30.165736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:30.166642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:30.167719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:30.168429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:30.172000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:30.219606 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 656/1343] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:29 2022] POST /identity/v3/users => generated 283 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:30.221462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:30.254426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0029d0-67f4-4149-8fa4-910e3653df08 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:30.258439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:30.273377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:30.274008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:30.274283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:30.274550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:30.276952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:30.281422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:30.304652 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 657/1344] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:30.308790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:30.330092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb489a-526a-40cd-af56-a673bfab58e1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:30.355792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:30.356811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:30.357301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:30.357753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:30.359454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:30.364568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:30.370692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f7a52-189b-4710-adcf-4a45ff36fb18 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:30.392597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803714ea-e109-491b-ba7a-cea37e53a686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:30.393983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803714ea-e109-491b-ba7a-cea37e53a686 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:30.394441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803714ea-e109-491b-ba7a-cea37e53a686 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:30.394885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803714ea-e109-491b-ba7a-cea37e53a686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:30.403728 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 658/1345] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:30 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:30.406129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:30.424819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803714ea-e109-491b-ba7a-cea37e53a686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:30.428777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803714ea-e109-491b-ba7a-cea37e53a686 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:30.442531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870d2ada-9792-4cf1-aea1-d01596d9e6a2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:30.454285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:30.455370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:30.456078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:30.465802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] PATH_INFO: `/v3/projects/4b303de821ac45068740a37163b8e0bd/users/82394a7488864c8f8cabfc5769018404/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:30.474373 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 688/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:30.502922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b303de821ac45068740a37163b8e0bd, user_id=82394a7488864c8f8cabfc5769018404, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:30.506039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:30.553122 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 659/1347] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:30 2022] PUT /identity/v3/projects/4b303de821ac45068740a37163b8e0bd/users/82394a7488864c8f8cabfc5769018404/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:30.556528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64581c1-f498-4428-adba-92b945d0836b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:30.557398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64581c1-f498-4428-adba-92b945d0836b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:30.557817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64581c1-f498-4428-adba-92b945d0836b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:30.610084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a64581c1-f498-4428-adba-92b945d0836b None None] MFA Rules not processed for user `82394a7488864c8f8cabfc5769018404`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:30.612243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a64581c1-f498-4428-adba-92b945d0836b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:30.643391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:30.645452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:30.780156 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 689/1348] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:30.781760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64581c1-f498-4428-adba-92b945d0836b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:30.824321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8866cece-7ea1-4dfb-92c7-d6bdc1656467 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:30.846871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:30.847700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:30.847981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:30.848249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] PATH_INFO: `/v3/projects/79e9ffd0be894aca9b7add142f4a54e2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:30.849765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e9ffd0be894aca9b7add142f4a54e2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:30.856039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:30.880461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:30.960555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64581c1-f498-4428-adba-92b945d0836b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:30.987629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e9ffd0be894aca9b7add142f4a54e2', 'request_id': 'req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.004971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e9ffd0be894aca9b7add142f4a54e2', 'request_id': 'req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.016891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e9ffd0be894aca9b7add142f4a54e2', 'request_id': 'req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.034702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] The token cache is being invalidate because project 79e9ffd0be894aca9b7add142f4a54e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:31.035491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.037627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e3ffd0b6-0b6a-4c64-93b2-87865e21c2eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.045276 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 660/1349] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:30 2022] DELETE /identity/v3/projects/79e9ffd0be894aca9b7add142f4a54e2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:31.051325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16b5b-6c19-44e8-b2b4-31cb50163d9c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.051620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16b5b-6c19-44e8-b2b4-31cb50163d9c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.051876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16b5b-6c19-44e8-b2b4-31cb50163d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.106016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:31.117259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b2f16b5b-6c19-44e8-b2b4-31cb50163d9c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:31.120444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b2f16b5b-6c19-44e8-b2b4-31cb50163d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:31.125368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:31.127777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.128389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.128674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.129324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:31.131955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:31.142282 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 661/1350] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:31.152598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.152881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.153163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.196805 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 690/1351] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:30 2022] POST /identity/v3/projects => generated 363 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:31.200187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:31.201359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:31.201528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:31.208449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:31.370826 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 662/1352] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:31.371695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:31.373245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:31.400261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e14cc2-26b3-459b-a054-d6e1bcc32eb4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:31.404988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e1b89-7c2f-400a-9581-13845124bdf1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:31.406201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:31.406974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.407542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.407848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.408526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:31.411353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:31.417857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:31.418943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.420652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.421970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] PATH_INFO: `/v3/projects/7bf497a4063c4290b224f38ed5769852` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.425921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bf497a4063c4290b224f38ed5769852)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:31.431792 dv-cinder-master-3-smb devstack@keystone.service[81372]: WARNING py.warnings [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 00:55:31.431792 dv-cinder-master-3-smb devstack@keystone.service[81372]: warnings.warn(msg) Oct 23 00:55:31.431792 dv-cinder-master-3-smb devstack@keystone.service[81372]: Oct 23 00:55:31.434034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:31.464724 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 663/1353] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:31 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:31.465791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:31.488785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9f9aaa-e89b-4b0d-9be8-bee5edafb806 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:31.504961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:31.505595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.505870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.507663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.508372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:31.508733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:31.511157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:31.545944 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 664/1354] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:31.547764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:31.574270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deebe90-6a00-4ced-96d7-4ca1b3ee72f5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:31.581204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bf497a4063c4290b224f38ed5769852', 'request_id': 'req-e82e7bb6-1cc7-41bc-be91-715869aadde7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.602142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bf497a4063c4290b224f38ed5769852', 'request_id': 'req-e82e7bb6-1cc7-41bc-be91-715869aadde7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.637852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bf497a4063c4290b224f38ed5769852', 'request_id': 'req-e82e7bb6-1cc7-41bc-be91-715869aadde7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.674167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] The token cache is being invalidate because project 7bf497a4063c4290b224f38ed5769852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:31.677920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.681931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.685839 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 691/1355] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:31 2022] DELETE /identity/v3/projects/7bf497a4063c4290b224f38ed5769852 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:31.692339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:31.695265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:31.727679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:31.728587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.729066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.729436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.730293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:31.734245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:31.740289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:31.758746 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 665/1356] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:31 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:31.761786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:31.767878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:31.794531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79939a6e-b847-4bb9-b7d2-c0fc0631a495 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:31.812480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:31.813415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.813995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.815649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] PATH_INFO: `/v3/users/8abe39e55ff745acbe0c075af6cc4fc2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.818357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82e7bb6-1cc7-41bc-be91-715869aadde7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:31.818615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8abe39e55ff745acbe0c075af6cc4fc2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:31.823221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:31.824830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:31.825632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:31.825992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:31.826337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:31.827276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:31.831519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:31.845839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:31.877612 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 692/1357] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:31 2022] POST /identity/v3/projects => generated 387 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:31.878766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:31.881226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:31.936633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8abe39e55ff745acbe0c075af6cc4fc2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.940763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8abe39e55ff745acbe0c075af6cc4fc2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.959462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8abe39e55ff745acbe0c075af6cc4fc2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.979906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8abe39e55ff745acbe0c075af6cc4fc2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.982470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8abe39e55ff745acbe0c075af6cc4fc2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:31.991934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd418c7c-658a-4f5a-9000-36e4fb7e8052 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:31.999384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.000802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.001087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.002309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] PATH_INFO: `/v3/projects/7e6ad7b533bf497ab08b3350f39b003b/users/88767e0f827244f7ac6d7361f1bf74a5/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.011575 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 666/1358] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:31 2022] DELETE /identity/v3/users/8abe39e55ff745acbe0c075af6cc4fc2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:32.012961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:32.016013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:32.046345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ad7b533bf497ab08b3350f39b003b, user_id=88767e0f827244f7ac6d7361f1bf74a5, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:32.057591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:32.087639 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 693/1359] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:31 2022] PUT /identity/v3/projects/7e6ad7b533bf497ab08b3350f39b003b/users/88767e0f827244f7ac6d7361f1bf74a5/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:32.091961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:32.098387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:32.130167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974107e-4ca4-48e7-8c39-b79bbe6c6733 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:32.232802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a59929-6b44-403a-ae33-d9120f7d2f9d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:32.240650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.241960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.242304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.242640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.244146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:32.246206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:32.260438 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 694/1360] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:55:32.261216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.263220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.263430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.263732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.265155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:32.265973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:32.268859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:32.282180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f26b6dd-7b1b-4a51-a1fb-9a6f607f25fb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:32.290267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba531da9-6ba4-4276-aa32-cc075ab87446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.292113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba531da9-6ba4-4276-aa32-cc075ab87446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.292635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba531da9-6ba4-4276-aa32-cc075ab87446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.293668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba531da9-6ba4-4276-aa32-cc075ab87446 admin admin] PATH_INFO: `/v3/projects/7e6ad7b533bf497ab08b3350f39b003b/users/88767e0f827244f7ac6d7361f1bf74a5/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.297192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba531da9-6ba4-4276-aa32-cc075ab87446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e6ad7b533bf497ab08b3350f39b003b, user_id=88767e0f827244f7ac6d7361f1bf74a5, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:32.301868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba531da9-6ba4-4276-aa32-cc075ab87446 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:32.320292 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 667/1361] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:32 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:32.321744 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 695/1362] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:32 2022] PUT /identity/v3/projects/7e6ad7b533bf497ab08b3350f39b003b/users/88767e0f827244f7ac6d7361f1bf74a5/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:32.325724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:32.335678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:32.347198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a68c0-8874-44f8-9573-a62592ed7ffd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.348233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a68c0-8874-44f8-9573-a62592ed7ffd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.349918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538a68c0-8874-44f8-9573-a62592ed7ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.352452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103c5a7-a53b-4c13-b5a8-5538e7aec403 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:32.457766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-538a68c0-8874-44f8-9573-a62592ed7ffd None None] MFA Rules not processed for user `88767e0f827244f7ac6d7361f1bf74a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:32.461919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-538a68c0-8874-44f8-9573-a62592ed7ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:32.522429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.523752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.524137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.524986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.526859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:32.529286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:32.553516 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 668/1363] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:32.558716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:32.579424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af4221c-1cfe-4d4e-85c7-2408deaecef2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:32.584624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:32.599592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.600337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.600697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.601950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.603341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:32.609275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:32.640707 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 696/1364] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:32.648147 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 669/1365] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:32 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:32.652233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a68c0-8874-44f8-9573-a62592ed7ffd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:32.656162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:32.663394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:32.685017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538a68c0-8874-44f8-9573-a62592ed7ffd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:32.698305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7918cc5-2461-41b9-911b-54fe52726cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.699191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7918cc5-2461-41b9-911b-54fe52726cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.701434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7918cc5-2461-41b9-911b-54fe52726cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.701822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7918cc5-2461-41b9-911b-54fe52726cad admin admin] PATH_INFO: `/v3/projects/0af8ea27331940d28594dc5cc89e03fe/users/dc342138236f41e6b2825c29eaf8b5c1/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.749238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7918cc5-2461-41b9-911b-54fe52726cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0af8ea27331940d28594dc5cc89e03fe, user_id=dc342138236f41e6b2825c29eaf8b5c1, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:32.752101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7918cc5-2461-41b9-911b-54fe52726cad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:32.776786 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 697/1366] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:32 2022] PUT /identity/v3/projects/0af8ea27331940d28594dc5cc89e03fe/users/dc342138236f41e6b2825c29eaf8b5c1/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:32.789078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf92af-6a49-49db-b9c0-583b4d8dfe73 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:32.789366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec071ef-e518-4fe1-a8c9-e77318ec2b1d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.789747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec071ef-e518-4fe1-a8c9-e77318ec2b1d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.790214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec071ef-e518-4fe1-a8c9-e77318ec2b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:32.863680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4ec071ef-e518-4fe1-a8c9-e77318ec2b1d None None] MFA Rules not processed for user `dc342138236f41e6b2825c29eaf8b5c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:32.865231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4ec071ef-e518-4fe1-a8c9-e77318ec2b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:32.897150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:32.897785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:32.898057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:32.898321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:33.013511 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 698/1367] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:33.018415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec071ef-e518-4fe1-a8c9-e77318ec2b1d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:33.030955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:33.041581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:33.043126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:33.062357 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 670/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:32 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:33.234980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec071ef-e518-4fe1-a8c9-e77318ec2b1d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:33.247985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:33.248418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:33.248632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:33.248904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:33.249546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:33.251500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:33.274462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:33.295425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b02b4b-13e0-4c3e-939a-c8bdc05b0427 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:33.327540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:33.330212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:33.332891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:33.333273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:33.340235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:33.358498 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 699/1369] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:33 2022] POST /identity/v3/projects => generated 387 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:55:33.363107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:33.434076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:33.508470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d27aa-aab8-4616-80a2-4c4e04fdaadd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:33.533681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:33.534329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:33.534606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:33.534876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:33.535715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:33.537490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:33.567465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:33.574747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:33.600965 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 671/1370] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:55:33.618889 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 700/1371] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:33 2022] POST /identity/v3/users => generated 300 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:55:33.622921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:33.637873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6398c7f-32e1-4f25-b73c-d579fbebf686 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:33.648185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:33.648941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:33.649267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:33.649727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:33.650461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:33.652343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:33.684751 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 672/1372] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:33.688808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:33.710511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325bfe2-d743-4f33-b85a-beba9533f935 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:33.724985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:33.725674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:33.725950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:33.726221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] PATH_INFO: `/v3/projects/7ad29830bfe94a7d848c974f4d483433/users/a1879bebf5f44821bb3ea142474a461e/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:33.757023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad29830bfe94a7d848c974f4d483433, user_id=a1879bebf5f44821bb3ea142474a461e, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:33.759968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:33.788207 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 701/1373] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:33 2022] PUT /identity/v3/projects/7ad29830bfe94a7d848c974f4d483433/users/a1879bebf5f44821bb3ea142474a461e/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:33.792284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:33.812816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5199ec86-98ff-4725-acbb-f97dec7c2aa0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:33.964272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:33.965537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:33.965817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:33.968376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:33.970512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:33.974724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:34.006495 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 673/1374] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:33 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:55:34.010979 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:34.026835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:34.032379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:34.032810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df97100-c224-431c-b127-487520bc7e4b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:34.047684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d50a6-c294-4dc2-a6ba-c4aad2cbd8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:34.049849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d50a6-c294-4dc2-a6ba-c4aad2cbd8cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.050396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d50a6-c294-4dc2-a6ba-c4aad2cbd8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.052063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d50a6-c294-4dc2-a6ba-c4aad2cbd8cb admin admin] PATH_INFO: `/v3/projects/7ad29830bfe94a7d848c974f4d483433/users/a1879bebf5f44821bb3ea142474a461e/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.058515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d50a6-c294-4dc2-a6ba-c4aad2cbd8cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ad29830bfe94a7d848c974f4d483433, user_id=a1879bebf5f44821bb3ea142474a461e, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:34.068712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d50a6-c294-4dc2-a6ba-c4aad2cbd8cb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:34.071456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f990db9-8686-4b15-81e0-f69c060b7bcb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:34.088519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:34.090475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.095425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.095939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] PATH_INFO: `/v3/projects/80bd8dbd74ed46cfb932b8e4f6e88585` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.103195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80bd8dbd74ed46cfb932b8e4f6e88585)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:34.110158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:34.122118 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 702/1375] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:34 2022] PUT /identity/v3/projects/7ad29830bfe94a7d848c974f4d483433/users/a1879bebf5f44821bb3ea142474a461e/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:34.125306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.127073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.127374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.194321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] MFA Rules not processed for user `a1879bebf5f44821bb3ea142474a461e`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:34.197665 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:34.228895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80bd8dbd74ed46cfb932b8e4f6e88585', 'request_id': 'req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.242758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80bd8dbd74ed46cfb932b8e4f6e88585', 'request_id': 'req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.254320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80bd8dbd74ed46cfb932b8e4f6e88585', 'request_id': 'req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.274213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] The token cache is being invalidate because project 80bd8dbd74ed46cfb932b8e4f6e88585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:34.274948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.275977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1908c384-b2cf-4b13-bfd3-b40fd9dbac23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.279248 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 674/1376] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:34 2022] DELETE /identity/v3/projects/80bd8dbd74ed46cfb932b8e4f6e88585 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:34.285291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e4a23-bd4f-448a-a669-13153b22dc28 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.287431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e4a23-bd4f-448a-a669-13153b22dc28 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.288981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e4a23-bd4f-448a-a669-13153b22dc28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.323064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:34.346791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-818e4a23-bd4f-448a-a669-13153b22dc28 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:34.348839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-818e4a23-bd4f-448a-a669-13153b22dc28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:34.406613 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 703/1377] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:34.408786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:34.411648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:34.504415 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 675/1378] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:34.520363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6055a84-fbdd-4f59-854a-17fc5fb42a2b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.520700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6055a84-fbdd-4f59-854a-17fc5fb42a2b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.521620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6055a84-fbdd-4f59-854a-17fc5fb42a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.530927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3680913-391f-489b-8d39-222bc62b21b1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:34.544181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:34.544861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.545143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.545418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.568623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:34.569233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e6055a84-fbdd-4f59-854a-17fc5fb42a2b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:34.570078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:34.570706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e6055a84-fbdd-4f59-854a-17fc5fb42a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:34.593229 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 676/1379] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:34.594580 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 704/1380] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:34 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:34.606821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6055a84-fbdd-4f59-854a-17fc5fb42a2b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:34.661579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6055a84-fbdd-4f59-854a-17fc5fb42a2b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:34.711085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:34.712510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:34.712682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:34.713159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.713444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.713720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.714384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:34.714746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:34.716290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:34.734207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7d491-f127-4158-be77-06bc50b4c628 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:34.749144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:34.749871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.750074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.750342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] PATH_INFO: `/v3/users/dc342138236f41e6b2825c29eaf8b5c1` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.752091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc342138236f41e6b2825c29eaf8b5c1)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:34.754365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:34.765552 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 677/1381] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:34.769872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:34.770102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:34.792678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f7dd8-be04-4757-9e91-1876aae6fc67 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:34.808951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:34.810478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.811629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.812197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:34.812981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:34.815591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:34.871662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc342138236f41e6b2825c29eaf8b5c1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.875022 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 678/1382] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:34 2022] POST /identity/v3/projects => generated 357 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:34.881464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:34.889114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc342138236f41e6b2825c29eaf8b5c1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.912177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc342138236f41e6b2825c29eaf8b5c1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.936570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc342138236f41e6b2825c29eaf8b5c1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.937530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6aa467f2-45c5-4604-b2b4-44c1015fdd17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc342138236f41e6b2825c29eaf8b5c1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:34.949370 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 705/1383] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:34 2022] DELETE /identity/v3/users/dc342138236f41e6b2825c29eaf8b5c1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:34.953825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:34.954125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:34.954396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:35.012215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeff527-4770-4dc2-a773-3afdf2993f86 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:35.024202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:35.028670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:35.131545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:35.132386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:35.132763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:35.133136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:35.133874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:35.135954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:35.188988 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 706/1384] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:35.194401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:35.203969 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 679/1385] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:34 2022] POST /identity/v3/users => generated 279 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:35.206015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:35.207617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:35.241410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d32c7d-f2c5-48c4-8323-04f338277e6d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:35.255268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:35.257846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:35.258178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:35.258461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:35.259143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:35.263996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:35.297324 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 707/1386] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:35.299323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:35.318821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb041b-29ce-4f88-9849-f440c61a80c3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:35.340841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9c54c-8005-4029-b6a7-d0d347c63077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:35.341484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c54c-8005-4029-b6a7-d0d347c63077 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:35.341769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c54c-8005-4029-b6a7-d0d347c63077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:35.342041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9c54c-8005-4029-b6a7-d0d347c63077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:35.342700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9c54c-8005-4029-b6a7-d0d347c63077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:35.344330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9c54c-8005-4029-b6a7-d0d347c63077 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:35.365827 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 680/1387] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:35 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:35.366494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:35.378672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c4500-251b-4023-98af-c1d6887a4860 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:35.403828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:35.404540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:35.404820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:35.405110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] PATH_INFO: `/v3/projects/6b0df7794db74f469bd8e7e52a5ceb67/users/776a1fde3f9e47bf8ca74f75e6693f19/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:35.445786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b0df7794db74f469bd8e7e52a5ceb67, user_id=776a1fde3f9e47bf8ca74f75e6693f19, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:35.453269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:35.487576 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 708/1388] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:35 2022] PUT /identity/v3/projects/6b0df7794db74f469bd8e7e52a5ceb67/users/776a1fde3f9e47bf8ca74f75e6693f19/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:35.500173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ac24c-0bd0-495a-9e02-fcb8eacdbb34 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:35.500543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ac24c-0bd0-495a-9e02-fcb8eacdbb34 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:35.500825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ac24c-0bd0-495a-9e02-fcb8eacdbb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:35.537162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:35.539366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:35.605480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5b7ac24c-0bd0-495a-9e02-fcb8eacdbb34 None None] MFA Rules not processed for user `776a1fde3f9e47bf8ca74f75e6693f19`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:35.608426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b7ac24c-0bd0-495a-9e02-fcb8eacdbb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:35.749499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:35.782146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec2889-94b6-47db-8cff-1eb03c11e5cc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:35.828657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:35.829550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:35.830511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:35.830951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:35.909658 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 681/1389] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:35.914153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ac24c-0bd0-495a-9e02-fcb8eacdbb34 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:36.014354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ac24c-0bd0-495a-9e02-fcb8eacdbb34 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:36.031218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:36.032261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.033328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.033569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:36.033791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.034857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:36.035554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:36.042375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:36.063297 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 709/1390] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:36.098258 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 682/1391] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:35 2022] POST /identity/v3/projects => generated 355 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:55:36.099151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:36.115035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793ca87-443b-417b-b180-74e94c600e18 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:36.257505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:36.258287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.258287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.258508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.259009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:36.261390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:36.308448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 710/1392] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:36 2022] POST /identity/v3/users => generated 284 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:36.313747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:36.339417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88574bd3-d4d0-43c2-8685-21eeffc01a7d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:36.347532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c0c730-cec4-40e2-b2dc-e417d6971891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:36.348661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0c730-cec4-40e2-b2dc-e417d6971891 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.349178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0c730-cec4-40e2-b2dc-e417d6971891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.349503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c0c730-cec4-40e2-b2dc-e417d6971891 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.350215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0c730-cec4-40e2-b2dc-e417d6971891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:36.352292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c0c730-cec4-40e2-b2dc-e417d6971891 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:36.379165 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 683/1393] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:36.382136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:36.398028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6badecb-b53e-41e5-95c7-7d7286bb35c5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:36.411184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:36.411879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.412162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.412471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] PATH_INFO: `/v3/projects/a07e4adadc0e4c42a32b6653abe8d201/users/804efaafbc3e47bfae1e6d193ccde3a3/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.449291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07e4adadc0e4c42a32b6653abe8d201, user_id=804efaafbc3e47bfae1e6d193ccde3a3, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:36.453820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:36.494728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7d3a-faab-40ca-aa1c-126c0e3ddaaf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.496953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7d3a-faab-40ca-aa1c-126c0e3ddaaf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.497989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb7d3a-faab-40ca-aa1c-126c0e3ddaaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.501301 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 711/1394] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:36 2022] PUT /identity/v3/projects/a07e4adadc0e4c42a32b6653abe8d201/users/804efaafbc3e47bfae1e6d193ccde3a3/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:36.507252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:36.565014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1cdb7d3a-faab-40ca-aa1c-126c0e3ddaaf None None] MFA Rules not processed for user `902cd162a23540d7b458a9f8b432b6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:36.568086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1cdb7d3a-faab-40ca-aa1c-126c0e3ddaaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:36.643685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:36.709270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54cc183-4fd4-4e4a-830f-e989ce213f4c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:36.715225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:36.716804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.717063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.717446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.718473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:36.721138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:36.727484 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 684/1395] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:36.738008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.738358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.738755 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.739292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:36.746358 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 712/1396] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:36 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:36.754848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:36.788086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c04a8-6930-4149-af79-10cee6fda130 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:36.794284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] MFA Rules not processed for user `902cd162a23540d7b458a9f8b432b6c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:36.801581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:36.852102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369b764-fe07-43ae-8fd1-41e2f6c9ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:36.852102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369b764-fe07-43ae-8fd1-41e2f6c9ffa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.852102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369b764-fe07-43ae-8fd1-41e2f6c9ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.852102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369b764-fe07-43ae-8fd1-41e2f6c9ffa4 admin admin] PATH_INFO: `/v3/projects/a07e4adadc0e4c42a32b6653abe8d201/users/804efaafbc3e47bfae1e6d193ccde3a3/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.852735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369b764-fe07-43ae-8fd1-41e2f6c9ffa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a07e4adadc0e4c42a32b6653abe8d201, user_id=804efaafbc3e47bfae1e6d193ccde3a3, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:36.852735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369b764-fe07-43ae-8fd1-41e2f6c9ffa4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:36.853808 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 685/1397] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:36.857964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:36.868069 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 713/1398] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:36 2022] PUT /identity/v3/projects/a07e4adadc0e4c42a32b6653abe8d201/users/804efaafbc3e47bfae1e6d193ccde3a3/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:36.881475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:36.884658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:36.884977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:36.885256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:36.996481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] MFA Rules not processed for user `804efaafbc3e47bfae1e6d193ccde3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:36.998051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:37.107214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6477cd5-1809-4066-8a16-f770be2d0b57 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:37.244751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:37.245394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:37.245699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:37.245941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:37.304932 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 714/1399] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:36 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:37.318455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:37.320588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:37.440032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:37.442067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:37.464791 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 686/1400] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:37.500794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d319a1-c9f6-481b-a94c-ad8bc165a885 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:37.512254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:37.512936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:37.513219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:37.513493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:37.539422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:37.544878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:37.575120 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 715/1401] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:37.678459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:37.682779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:37.702250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:37.804966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:37.811924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:37.832903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4619c4ca-4ed8-4ae8-8ae2-c0465885a647 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:37.850480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec77820-9250-4b01-9850-8d0b7cfff84c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:37.855027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec77820-9250-4b01-9850-8d0b7cfff84c service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:37.856510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec77820-9250-4b01-9850-8d0b7cfff84c service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:37.856510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec77820-9250-4b01-9850-8d0b7cfff84c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:37.867195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ee775-7291-4b88-9232-b451eafa0545 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:37.877205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:37.878392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:37.878818 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:37.879233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] PATH_INFO: `/v3/users/82394a7488864c8f8cabfc5769018404` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:37.880653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82394a7488864c8f8cabfc5769018404)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:37.886156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:37.985568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec77820-9250-4b01-9850-8d0b7cfff84c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:37.987465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec77820-9250-4b01-9850-8d0b7cfff84c service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:38.000331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 716/1402] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:38.005299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82394a7488864c8f8cabfc5769018404'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:38.006124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82394a7488864c8f8cabfc5769018404'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:38.023550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82394a7488864c8f8cabfc5769018404'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:38.118042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82394a7488864c8f8cabfc5769018404'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:38.119172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82394a7488864c8f8cabfc5769018404'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:38.150366 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 687/1403] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:37 2022] DELETE /identity/v3/users/82394a7488864c8f8cabfc5769018404 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:38.155731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867e9f7-994c-4c89-bea8-8ddd820d647f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:38.156084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867e9f7-994c-4c89-bea8-8ddd820d647f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:38.156390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7867e9f7-994c-4c89-bea8-8ddd820d647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:38.205974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7867e9f7-994c-4c89-bea8-8ddd820d647f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:38.208871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7867e9f7-994c-4c89-bea8-8ddd820d647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:38.228916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:38.337397 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 717/1404] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:38.385856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:38.387401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:38.547432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6278b-9178-4ac0-9f5a-9a69eff734a6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:38.557402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2117f878-2dec-46d7-b31d-7c96c9f1f834 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:38.558099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117f878-2dec-46d7-b31d-7c96c9f1f834 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:38.558383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117f878-2dec-46d7-b31d-7c96c9f1f834 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:38.558660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2117f878-2dec-46d7-b31d-7c96c9f1f834 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:38.583277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117f878-2dec-46d7-b31d-7c96c9f1f834 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:38.584758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2117f878-2dec-46d7-b31d-7c96c9f1f834 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:38.612598 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 688/1405] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:38.774981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208628e-a0f4-4038-b68c-61a072dbf84d None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:38.776547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208628e-a0f4-4038-b68c-61a072dbf84d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:38.776999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208628e-a0f4-4038-b68c-61a072dbf84d None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:38.782060 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 718/1406] 10.222.0.39 () {54 vars in 912 bytes} [Sun Oct 23 00:55:38 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:55:38.789999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:38.790503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:38.790589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:38.912063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] MFA Rules not processed for user `bb5f82d3cfc14eacbe4cc40bcd0dfb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:38.914783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:38.942725 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 689/1407] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:55:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:39.134939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208628e-a0f4-4038-b68c-61a072dbf84d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:39.137002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8208628e-a0f4-4038-b68c-61a072dbf84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:39.163513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208628e-a0f4-4038-b68c-61a072dbf84d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:39.170873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:39.172279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:39.177793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734718e2-d8ef-4a0d-ba34-9a4c45a14326 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:39.178427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734718e2-d8ef-4a0d-ba34-9a4c45a14326 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:39.178707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734718e2-d8ef-4a0d-ba34-9a4c45a14326 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:39.178980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734718e2-d8ef-4a0d-ba34-9a4c45a14326 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:39.210811 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734718e2-d8ef-4a0d-ba34-9a4c45a14326 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:39.212017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734718e2-d8ef-4a0d-ba34-9a4c45a14326 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:39.217404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178928a3-46b0-4a6d-814a-a0f648ea22a3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:39.227094 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 719/1408] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:39.227744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:39.228432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:39.228892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:39.228989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] PATH_INFO: `/v3/projects/0af8ea27331940d28594dc5cc89e03fe` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:39.241256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0af8ea27331940d28594dc5cc89e03fe)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:39.243238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:39.251230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:39.300431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0a78-1516-49b7-8d92-fddeda1cbaa3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:39.300742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0a78-1516-49b7-8d92-fddeda1cbaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:39.301172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585d0a78-1516-49b7-8d92-fddeda1cbaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:39.348825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-585d0a78-1516-49b7-8d92-fddeda1cbaa3 None None] MFA Rules not processed for user `11b0f829f5654010b2bd1a76888d7ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:39.358053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-585d0a78-1516-49b7-8d92-fddeda1cbaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:39.360212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0af8ea27331940d28594dc5cc89e03fe', 'request_id': 'req-b5bae753-62da-4571-b549-949d191c6a1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.377294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0af8ea27331940d28594dc5cc89e03fe', 'request_id': 'req-b5bae753-62da-4571-b549-949d191c6a1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.389173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0af8ea27331940d28594dc5cc89e03fe', 'request_id': 'req-b5bae753-62da-4571-b549-949d191c6a1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.406667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] The token cache is being invalidate because project 0af8ea27331940d28594dc5cc89e03fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:39.406969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.407847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.418350 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 690/1409] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:39 2022] DELETE /identity/v3/projects/0af8ea27331940d28594dc5cc89e03fe => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:39.419720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:39.421462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:39.541138 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 720/1410] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:39.542516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a036-f1cc-4281-9596-6dfb77b7984b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:39.542810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a036-f1cc-4281-9596-6dfb77b7984b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:39.549250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d20a036-f1cc-4281-9596-6dfb77b7984b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:39.633267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bae753-62da-4571-b549-949d191c6a1d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:39.639286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7d20a036-f1cc-4281-9596-6dfb77b7984b None None] MFA Rules not processed for user `28563be0db564f8ebcf765a78970ce8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:39.642362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7d20a036-f1cc-4281-9596-6dfb77b7984b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:39.646409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:39.647404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:39.647513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:39.647823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] PATH_INFO: `/v3/users/a1879bebf5f44821bb3ea142474a461e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:39.655767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1879bebf5f44821bb3ea142474a461e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:39.661648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:39.796912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1879bebf5f44821bb3ea142474a461e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.798968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1879bebf5f44821bb3ea142474a461e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.834701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1879bebf5f44821bb3ea142474a461e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.842554 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 721/1411] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:39.856871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:39.857119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:39.857340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:39.874572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1879bebf5f44821bb3ea142474a461e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.875004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1879bebf5f44821bb3ea142474a461e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:39.903464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] MFA Rules not processed for user `28563be0db564f8ebcf765a78970ce8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:39.909599 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 691/1412] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:39 2022] DELETE /identity/v3/users/a1879bebf5f44821bb3ea142474a461e => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:39.909770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:39.979101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:40.048886 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 722/1413] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:39 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:40.165318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:40.167123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:40.190975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:40.316456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:40.323308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:40.392527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a382599-8f21-4110-b87d-83e9bc3d4aff admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:40.401178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397061f7-108f-4348-b07f-750632483bc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:40.402086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397061f7-108f-4348-b07f-750632483bc8 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:40.402558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397061f7-108f-4348-b07f-750632483bc8 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:40.403105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397061f7-108f-4348-b07f-750632483bc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:40.406187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-397061f7-108f-4348-b07f-750632483bc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:40.425535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397061f7-108f-4348-b07f-750632483bc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:40.427598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397061f7-108f-4348-b07f-750632483bc8 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:40.446302 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 692/1414] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:40.514018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cfb64-473d-4110-b6ca-160fe2b37e95 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:40.514299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cfb64-473d-4110-b6ca-160fe2b37e95 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:40.516082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170cfb64-473d-4110-b6ca-160fe2b37e95 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:40.517825 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 693/1415] 10.222.0.39 () {54 vars in 912 bytes} [Sun Oct 23 00:55:40 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 00:55:40.527557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:40.528055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:40.528510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:40.586866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] MFA Rules not processed for user `bb5f82d3cfc14eacbe4cc40bcd0dfb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:40.589347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:40.597274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a6009-802b-45db-877f-94a071cbb645 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:40.611449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:40.612080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:40.612412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:40.612685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:40.617322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:40.659815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:40.663592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:40.704677 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 723/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:40.722943 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 694/1417] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:55:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:40.748227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:40.750043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:40.774567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2c29-55b8-4406-9880-ef0ba66b4d25 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:40.785215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:40.785849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:40.786273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:40.786557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:40.807763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:40.812587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:40.840949 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 724/1418] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:41.181034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:41.182495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:41.468860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc83b5c3-6526-4961-ae7f-5b35eab28f72 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:41.489152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:41.496486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:41.496943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:41.497743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] PATH_INFO: `/v3/projects/7ad29830bfe94a7d848c974f4d483433` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:41.505499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ad29830bfe94a7d848c974f4d483433)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:41.521830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:41.543664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:41.545193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:41.601197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d106c6-5536-42e2-8098-b94cf9454041 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:41.614448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:41.615351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:41.615891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:41.616382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] PATH_INFO: `/v3/projects/4b303de821ac45068740a37163b8e0bd` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:41.617745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b303de821ac45068740a37163b8e0bd)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:41.620026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:41.655557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ad29830bfe94a7d848c974f4d483433', 'request_id': 'req-82e02d88-ea10-477b-8a99-8cadb09d9ae1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.658234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:41.682059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ad29830bfe94a7d848c974f4d483433', 'request_id': 'req-82e02d88-ea10-477b-8a99-8cadb09d9ae1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.691130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ad29830bfe94a7d848c974f4d483433', 'request_id': 'req-82e02d88-ea10-477b-8a99-8cadb09d9ae1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.716906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] The token cache is being invalidate because project 7ad29830bfe94a7d848c974f4d483433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:41.717400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.727258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-82e02d88-ea10-477b-8a99-8cadb09d9ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.734516 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 695/1419] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:41 2022] DELETE /identity/v3/projects/7ad29830bfe94a7d848c974f4d483433 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:41.740979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:41.741257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:41.742869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:41.836368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b303de821ac45068740a37163b8e0bd', 'request_id': 'req-e4b6846e-28c9-4029-b626-a27b0638359d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.871202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b303de821ac45068740a37163b8e0bd', 'request_id': 'req-e4b6846e-28c9-4029-b626-a27b0638359d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.880226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b303de821ac45068740a37163b8e0bd', 'request_id': 'req-e4b6846e-28c9-4029-b626-a27b0638359d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.881365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] MFA Rules not processed for user `992874ab1b9c4fce90b46e659910d512`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:41.883530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:41.951801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] The token cache is being invalidate because project 4b303de821ac45068740a37163b8e0bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:41.953110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.963713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e4b6846e-28c9-4029-b626-a27b0638359d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:41.969648 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 725/1420] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:41 2022] DELETE /identity/v3/projects/4b303de821ac45068740a37163b8e0bd => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:41.972505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81e107-4198-40fe-8e82-d52d0bd7f0d8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:41.973045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81e107-4198-40fe-8e82-d52d0bd7f0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:41.973561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81e107-4198-40fe-8e82-d52d0bd7f0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:42.025157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8e81e107-4198-40fe-8e82-d52d0bd7f0d8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:42.043269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e81e107-4198-40fe-8e82-d52d0bd7f0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:42.184718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:42.221141 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 696/1421] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 486 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:42.230043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:42.230043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:42.278507 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 726/1422] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:42.287016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14400e01-5fdc-4788-bba6-677bb4a246ce None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:42.287405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14400e01-5fdc-4788-bba6-677bb4a246ce None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:42.287712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14400e01-5fdc-4788-bba6-677bb4a246ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:42.360434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-14400e01-5fdc-4788-bba6-677bb4a246ce None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:42.362882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-14400e01-5fdc-4788-bba6-677bb4a246ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:42.403489 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 727/1423] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:42.404493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14400e01-5fdc-4788-bba6-677bb4a246ce None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:42.440521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14400e01-5fdc-4788-bba6-677bb4a246ce None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:42.459896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:42.462327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:42.462465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:42.462729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:42.465541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:42.465857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:42.472270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:42.510023 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 728/1424] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:42.511121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:42.535539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d5350-2dbd-441a-865e-89c3f9e83c2d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:42.543085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5621f-5d6b-474b-91ce-aa6615e2aad3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:42.550401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:42.552150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:42.555266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:42.555911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:42.561997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:42.562960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:42.563996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:42.565431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:42.566210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:42.568322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] PATH_INFO: `/v3/users/88767e0f827244f7ac6d7361f1bf74a5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:42.570231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88767e0f827244f7ac6d7361f1bf74a5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:42.576927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:42.725033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88767e0f827244f7ac6d7361f1bf74a5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:42.726875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88767e0f827244f7ac6d7361f1bf74a5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:42.744007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88767e0f827244f7ac6d7361f1bf74a5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:42.769088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=1,keystone:DELETE=11,keystone:SELECT=88 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:42.773803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88767e0f827244f7ac6d7361f1bf74a5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:42.776712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88767e0f827244f7ac6d7361f1bf74a5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:42.807121 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 697/1425] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:42 2022] DELETE /identity/v3/users/88767e0f827244f7ac6d7361f1bf74a5 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:42.845344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:42.847775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:43.163439 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 729/1426] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:42 2022] POST /identity/v3/projects => generated 389 bytes in 652 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:43.166628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:43.169602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:43.172585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf2e46-6d92-437e-9b5b-a605abeee111 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:43.359502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc6c54-2120-41db-b9c5-a71a068133e4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:43.381802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:43.383615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:43.387844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:43.388733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:43.389581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:43.394433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:43.469316 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 730/1427] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:43 2022] POST /identity/v3/users => generated 295 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:43.473659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:43.491675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998f563-64d7-438c-8df7-3c3f89ae4c77 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:43.499957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb7c92-ad45-4120-9796-02fa598f47fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:43.505265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb7c92-ad45-4120-9796-02fa598f47fd service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:43.505654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb7c92-ad45-4120-9796-02fa598f47fd service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:43.505854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:43.505991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb7c92-ad45-4120-9796-02fa598f47fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:43.506485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:43.506765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:43.507039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:43.507693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:43.509337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:43.517643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-71eb7c92-ad45-4120-9796-02fa598f47fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:43.540680 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 731/1428] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:43.541808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:43.575046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6ef9c-d86f-4c21-a12d-570b510746d2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:43.583684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:43.584633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:43.585071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:43.585560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:43.586419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:43.588220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:43.615692 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 732/1429] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:43.619797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:43.665686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bccfd5-fd18-4d7a-98d1-594215e609a3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:43.681690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2382a9d9-7b42-495d-947b-1a714d5f1d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:43.682973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a9d9-7b42-495d-947b-1a714d5f1d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:43.683415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a9d9-7b42-495d-947b-1a714d5f1d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:43.684133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2382a9d9-7b42-495d-947b-1a714d5f1d66 admin admin] PATH_INFO: `/v3/projects/5895d8289ac644b4afb5c2ac85c85c54/users/b94d9e7c3f974d6884cca1f65bed2b04/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:43.748009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382a9d9-7b42-495d-947b-1a714d5f1d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5895d8289ac644b4afb5c2ac85c85c54, user_id=b94d9e7c3f974d6884cca1f65bed2b04, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:43.756089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2382a9d9-7b42-495d-947b-1a714d5f1d66 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:43.806487 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 733/1430] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:43 2022] PUT /identity/v3/projects/5895d8289ac644b4afb5c2ac85c85c54/users/b94d9e7c3f974d6884cca1f65bed2b04/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:43.808112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d0797-d94f-4f8b-ad93-d6c81095ed9c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:43.808653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d0797-d94f-4f8b-ad93-d6c81095ed9c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:43.809079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d0797-d94f-4f8b-ad93-d6c81095ed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:43.817281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb7c92-ad45-4120-9796-02fa598f47fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:43.820574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb7c92-ad45-4120-9796-02fa598f47fd service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:43.850869 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 698/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 1006 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:55:43.856583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0d85f-ee62-4bdd-89da-dce6bda9bd7a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:43.856871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0d85f-ee62-4bdd-89da-dce6bda9bd7a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:43.857147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0d85f-ee62-4bdd-89da-dce6bda9bd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:43.881498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-578d0797-d94f-4f8b-ad93-d6c81095ed9c None None] MFA Rules not processed for user `28563be0db564f8ebcf765a78970ce8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:43.882736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-578d0797-d94f-4f8b-ad93-d6c81095ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:43.956019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4ea0d85f-ee62-4bdd-89da-dce6bda9bd7a None None] MFA Rules not processed for user `b94d9e7c3f974d6884cca1f65bed2b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:43.978049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4ea0d85f-ee62-4bdd-89da-dce6bda9bd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:44.004557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:44.090895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:44.188369 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 734/1432] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:44.280229 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 699/1433] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:43 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:44.285688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d0797-d94f-4f8b-ad93-d6c81095ed9c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:44.334504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ac0b6-b921-49cb-8271-db23bb77ff27 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:44.335027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ac0b6-b921-49cb-8271-db23bb77ff27 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:44.335626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927ac0b6-b921-49cb-8271-db23bb77ff27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:44.429444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-927ac0b6-b921-49cb-8271-db23bb77ff27 None None] MFA Rules not processed for user `959991a17c69459dbd665c2ae20095ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:44.430686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-927ac0b6-b921-49cb-8271-db23bb77ff27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:44.544139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d0797-d94f-4f8b-ad93-d6c81095ed9c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:44.578449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:44.585842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:44.586454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:44.588930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:44.589862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:44.597744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:44.683653 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 735/1434] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:44 2022] POST /identity/v3/projects => generated 389 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:44.687507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:44.690539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:44.725568 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 700/1435] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:44.726529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ac0b6-b921-49cb-8271-db23bb77ff27 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:44.757723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927ac0b6-b921-49cb-8271-db23bb77ff27 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:44.868206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6cf7fb-3f07-4b11-baee-980ea4bfdd87 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:44.879520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:44.881339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:44.882442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:44.882986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] PATH_INFO: `/v3/projects/7e6ad7b533bf497ab08b3350f39b003b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:44.884676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e6ad7b533bf497ab08b3350f39b003b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:44.888725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:44.934592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:44.935876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:44.936369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:44.936926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:44.938049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:44.940736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:44.995354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e6ad7b533bf497ab08b3350f39b003b', 'request_id': 'req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:45.010048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 701/1436] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:44 2022] POST /identity/v3/users => generated 301 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:45.015363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:45.024413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e6ad7b533bf497ab08b3350f39b003b', 'request_id': 'req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:45.032733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e6ad7b533bf497ab08b3350f39b003b', 'request_id': 'req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:45.058691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60b087-8d21-443c-a705-d10fd4aa4c11 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:45.063421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] The token cache is being invalidate because project 7e6ad7b533bf497ab08b3350f39b003b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:45.067461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:45.071113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6b0c99bd-9f8b-4f8d-9bd3-2f424df1844d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:45.085124 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 736/1437] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:44 2022] DELETE /identity/v3/projects/7e6ad7b533bf497ab08b3350f39b003b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:45.091049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.091969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.093078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.111097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:45.139334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:45.143469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:45.167776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:45.248442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:45.249129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.249411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.249692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.250326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:45.252553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:45.280998 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 702/1438] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:45.281502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:45.283243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:45.322479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01539fdc-766f-4ed9-9d7d-92b347369273 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:45.334674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0187a6-9de0-400b-a9b0-61c78f1ec244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:45.335956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0187a6-9de0-400b-a9b0-61c78f1ec244 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.336260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0187a6-9de0-400b-a9b0-61c78f1ec244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.336558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0187a6-9de0-400b-a9b0-61c78f1ec244 admin admin] PATH_INFO: `/v3/projects/c03a04af6f6b4fcf90525d89bb327974/users/8ca9787790e24b83bbeddfd35f61a6d2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.355558 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 737/1439] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:45.357127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:45.364095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:45.382292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0187a6-9de0-400b-a9b0-61c78f1ec244 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03a04af6f6b4fcf90525d89bb327974, user_id=8ca9787790e24b83bbeddfd35f61a6d2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:45.385027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0187a6-9de0-400b-a9b0-61c78f1ec244 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:45.422511 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 703/1440] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:45 2022] PUT /identity/v3/projects/c03a04af6f6b4fcf90525d89bb327974/users/8ca9787790e24b83bbeddfd35f61a6d2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:45.424365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b5bc1-e821-4964-999b-48fb321cd077 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.424819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b5bc1-e821-4964-999b-48fb321cd077 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.425240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b5bc1-e821-4964-999b-48fb321cd077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.475450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-295b5bc1-e821-4964-999b-48fb321cd077 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:45.478265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-295b5bc1-e821-4964-999b-48fb321cd077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:45.629700 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 704/1441] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:45.631931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b5bc1-e821-4964-999b-48fb321cd077 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:45.675492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b5bc1-e821-4964-999b-48fb321cd077 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:45.677895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78eb9ee-febd-43f6-ad9c-817f353df56e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:45.712484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:45.713162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.713445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.713724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.719990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:45.722033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:45.785432 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 705/1442] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:45 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:45.787195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:45.834717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de6e19d-d4d9-4873-a794-f7e9034ace7e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:45.873677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:45.874345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.877971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.878696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.879549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:45.881664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:45.883671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:45.901700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ba762-0e86-4b8e-a8d6-56d0f3eb7fa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:45.902342 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ba762-0e86-4b8e-a8d6-56d0f3eb7fa2 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.902634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ba762-0e86-4b8e-a8d6-56d0f3eb7fa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.906043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ba762-0e86-4b8e-a8d6-56d0f3eb7fa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.913856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7a8ba762-0e86-4b8e-a8d6-56d0f3eb7fa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:45.916444 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 706/1443] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:45.920336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:45.940360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e437c3-85f3-44cf-9feb-6dffa34d6c48 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:45.966916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:45.967623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:45.967910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:45.968183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] PATH_INFO: `/v3/projects/c03a04af6f6b4fcf90525d89bb327974/users/8ca9787790e24b83bbeddfd35f61a6d2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:45.976536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c03a04af6f6b4fcf90525d89bb327974, user_id=8ca9787790e24b83bbeddfd35f61a6d2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:45.979403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:46.014397 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 707/1444] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:45 2022] PUT /identity/v3/projects/c03a04af6f6b4fcf90525d89bb327974/users/8ca9787790e24b83bbeddfd35f61a6d2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:46.031073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:46.042903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:46.092351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ba762-0e86-4b8e-a8d6-56d0f3eb7fa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:46.097289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ba762-0e86-4b8e-a8d6-56d0f3eb7fa2 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:46.123042 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 738/1445] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 766 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:46.125679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bba4b4-fbaa-4d11-86aa-554cdb1e29d7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:46.126030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bba4b4-fbaa-4d11-86aa-554cdb1e29d7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:46.126813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bba4b4-fbaa-4d11-86aa-554cdb1e29d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:46.290563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6fed-aa98-4732-9aef-a862da21868b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:46.305644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc812e-f4c8-40d8-8e65-c4ed91f6893e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:46.306582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc812e-f4c8-40d8-8e65-c4ed91f6893e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:46.309153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc812e-f4c8-40d8-8e65-c4ed91f6893e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:46.312140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc812e-f4c8-40d8-8e65-c4ed91f6893e admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:46.316246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc812e-f4c8-40d8-8e65-c4ed91f6893e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:46.322502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc812e-f4c8-40d8-8e65-c4ed91f6893e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:46.343350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:46.348441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-33bba4b4-fbaa-4d11-86aa-554cdb1e29d7 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:46.353274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-33bba4b4-fbaa-4d11-86aa-554cdb1e29d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:46.391263 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 708/1446] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:46 2022] POST /identity/v3/projects => generated 375 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:46.391263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:46.391263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:46.391263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:46.474280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] MFA Rules not processed for user `8ca9787790e24b83bbeddfd35f61a6d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:46.476528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:46.549935 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 739/1447] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:55:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:46.551826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bba4b4-fbaa-4d11-86aa-554cdb1e29d7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:46.616405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bba4b4-fbaa-4d11-86aa-554cdb1e29d7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:46.743986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:46.744708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:46.744784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:46.745054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:46.746067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:46.747640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:46.778301 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 740/1448] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:46 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:46.779417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:46.790437 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 709/1449] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:46 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:46.802525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:46.804416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:46.838129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e769d-e25b-45e9-b873-a648f984a84f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:46.855147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:46.855774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:46.856056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:46.856353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:46.857051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:46.859455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:46.880455 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 741/1450] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:46.881345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:46.886597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:46.973632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:47.041294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c72c175-6836-4ec7-b5d3-b5ce081c839e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:47.054833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:47.055503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:47.056943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:47.057748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:47.089781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:47.091905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:47.121661 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 710/1451] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:46 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:55:47.127422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:47.161926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa72cc9-8cec-4e0c-8832-20950bbd6f98 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:47.165094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bd52dd-6555-4561-95b1-6abd8b9ab2f8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:47.186665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:47.188912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:47.189217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:47.190958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb609d-2d7a-4ed6-b65e-59d120817abd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:47.194562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb609d-2d7a-4ed6-b65e-59d120817abd service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:47.196224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb609d-2d7a-4ed6-b65e-59d120817abd service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:47.196510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb609d-2d7a-4ed6-b65e-59d120817abd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:47.203586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:47.204421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:47.213317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:47.272539 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 711/1452] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:47.275325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:47.304594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd50fb-428e-4864-877e-ea5512a93883 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:47.325160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:47.325863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:47.326143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:47.328360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] PATH_INFO: `/v3/projects/d311f0fc85544325bb9400f0efd54b8a/users/df7a46c9bf1f47ca8fe22d7fb5a38e40/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:47.352832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb609d-2d7a-4ed6-b65e-59d120817abd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:47.354448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb609d-2d7a-4ed6-b65e-59d120817abd service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:47.371189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d311f0fc85544325bb9400f0efd54b8a, user_id=df7a46c9bf1f47ca8fe22d7fb5a38e40, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:47.377657 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 742/1453] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 00:55:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:55:47.379318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:47.413915 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 712/1454] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:47 2022] PUT /identity/v3/projects/d311f0fc85544325bb9400f0efd54b8a/users/df7a46c9bf1f47ca8fe22d7fb5a38e40/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:47.430599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f1269-5d89-4692-b7b6-34a51434aa0f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:47.431197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f1269-5d89-4692-b7b6-34a51434aa0f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:47.431696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f1269-5d89-4692-b7b6-34a51434aa0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:47.555760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fc2f1269-5d89-4692-b7b6-34a51434aa0f None None] MFA Rules not processed for user `df7a46c9bf1f47ca8fe22d7fb5a38e40`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:47.557174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fc2f1269-5d89-4692-b7b6-34a51434aa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:47.604295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:47.804919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:47.929638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef3741c-a825-4c6d-b261-948e61b095d3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:47.931860 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 743/1455] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:47.937421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f1269-5d89-4692-b7b6-34a51434aa0f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:47.946924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:47.948972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:47.949265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:47.949540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] PATH_INFO: `/v3/users/b94d9e7c3f974d6884cca1f65bed2b04` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:47.952800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b94d9e7c3f974d6884cca1f65bed2b04)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:47.962230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:48.029170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f1269-5d89-4692-b7b6-34a51434aa0f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:48.059701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:48.062577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:48.062782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:48.063056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:48.065835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:48.071496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:48.178698 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 744/1456] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:47 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:48.180293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:48.185013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b94d9e7c3f974d6884cca1f65bed2b04'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:48.187835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b94d9e7c3f974d6884cca1f65bed2b04'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:48.245842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b94d9e7c3f974d6884cca1f65bed2b04'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:48.259258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:48.320990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b94d9e7c3f974d6884cca1f65bed2b04'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:48.324828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-acefa1da-7333-4adb-94e5-ac6216a04f8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b94d9e7c3f974d6884cca1f65bed2b04'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:48.386876 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 713/1457] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:47 2022] DELETE /identity/v3/users/b94d9e7c3f974d6884cca1f65bed2b04 => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:48.392035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:48.416516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:48.416859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:48.514310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:48.515576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:48.520765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87bdc94-7b36-41dd-b0a8-49c1c3c9ba65 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:48.781019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:48.781480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:48.786738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:48.787118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:48.791489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:48.797686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:48.859022 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 714/1458] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:48.882470 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 745/1459] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:48 2022] POST /identity/v3/users => generated 295 bytes in 700 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:48.884484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:48.885959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:48.936224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710fae1-3eb3-482b-a3e1-3d961ff81ccf None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:48.978868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:48.979960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:48.980642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:48.981082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:48.981236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:48.981826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:48.983032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:48.994620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:49.045672 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 715/1460] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:48 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:49.051676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:49.092244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f049e-bcd5-4fa4-a295-36c8037102d7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:49.128696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:49.129207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:49.129447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:49.129726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] PATH_INFO: `/v3/projects/a103b217545946d9860889815633d369/users/788e91ef45d94451bb4d20a530275e3a/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:49.136055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16023841-2c60-4790-b32c-1837045b5321 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:49.145413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:49.146663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:49.147431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:49.147918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:49.163816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:49.165814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:49.185553 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 746/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:49.226020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a103b217545946d9860889815633d369, user_id=788e91ef45d94451bb4d20a530275e3a, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:49.232978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:49.285667 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 716/1462] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:49 2022] PUT /identity/v3/projects/a103b217545946d9860889815633d369/users/788e91ef45d94451bb4d20a530275e3a/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:49.290169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:49.314882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c93161-4ecf-4464-9d9a-3e60465e0ec9 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:49.508848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:49.516125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db8c86-8b8a-4045-80c5-5beade444ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:49.518128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db8c86-8b8a-4045-80c5-5beade444ae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:49.518559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db8c86-8b8a-4045-80c5-5beade444ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:49.520547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db8c86-8b8a-4045-80c5-5beade444ae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:49.521380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db8c86-8b8a-4045-80c5-5beade444ae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:49.523792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db8c86-8b8a-4045-80c5-5beade444ae0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:49.539216 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 747/1463] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:49 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:49.543932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:49.579117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a2122-6454-45e0-9065-9100c50f2cc4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:49.592164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:49.597679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:49.597978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:49.599294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] PATH_INFO: `/v3/projects/a103b217545946d9860889815633d369/users/788e91ef45d94451bb4d20a530275e3a/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:49.602727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a103b217545946d9860889815633d369, user_id=788e91ef45d94451bb4d20a530275e3a, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:49.633960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:49.683742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275f1b-a19d-463a-8c29-ddc6fb327532 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:49.684216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275f1b-a19d-463a-8c29-ddc6fb327532 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:49.684518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd275f1b-a19d-463a-8c29-ddc6fb327532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:49.699811 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 717/1464] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:49 2022] PUT /identity/v3/projects/a103b217545946d9860889815633d369/users/788e91ef45d94451bb4d20a530275e3a/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:49.801852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bd275f1b-a19d-463a-8c29-ddc6fb327532 None None] MFA Rules not processed for user `788e91ef45d94451bb4d20a530275e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:49.804924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bd275f1b-a19d-463a-8c29-ddc6fb327532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:50.133094 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 748/1465] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:50.145708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:50.151392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:50.223893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:50.436186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292ac33-f13c-4806-8861-b984357ddbf2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:50.466789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe530c5-bce9-4169-8e72-1257c22c56cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:50.467429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe530c5-bce9-4169-8e72-1257c22c56cf service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:50.467709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe530c5-bce9-4169-8e72-1257c22c56cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:50.467980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe530c5-bce9-4169-8e72-1257c22c56cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:50.483951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bfe530c5-bce9-4169-8e72-1257c22c56cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:50.727421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe530c5-bce9-4169-8e72-1257c22c56cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:50.729003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe530c5-bce9-4169-8e72-1257c22c56cf service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:50.816854 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 718/1466] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 671 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:51.490327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:51.491265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:51.491627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:51.566004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] MFA Rules not processed for user `bb5f82d3cfc14eacbe4cc40bcd0dfb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:51.569077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:51.718905 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 749/1467] 10.222.0.43 () {58 vars in 1015 bytes} [Sun Oct 23 00:55:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:51.983726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d9a23-5188-439c-bbba-9ccef1470da5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:51.984618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d9a23-5188-439c-bbba-9ccef1470da5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:51.985110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d9a23-5188-439c-bbba-9ccef1470da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:52.055730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d61d9a23-5188-439c-bbba-9ccef1470da5 None None] MFA Rules not processed for user `992874ab1b9c4fce90b46e659910d512`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:52.057012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d61d9a23-5188-439c-bbba-9ccef1470da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:52.281748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:52.290627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:52.304412 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 719/1468] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:52.319758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458796a-bbbf-4a9b-804a-85a956bd6dc2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:52.327719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d9a23-5188-439c-bbba-9ccef1470da5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:52.332600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:52.333305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:52.333583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:52.333867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:52.353765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:52.356536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:52.367342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d9a23-5188-439c-bbba-9ccef1470da5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:52.373541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:52.374700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:52.375452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:52.376114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:52.440659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:52.448876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:52.451177 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 750/1469] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:52.478374 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 720/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:52.603309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:52.606064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:52.937269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306c07f7-cb94-4cfb-83b6-4cd6ffd676a1 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:52.956160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:52.957321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:52.957657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:52.958109 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] PATH_INFO: `/v3/projects/5895d8289ac644b4afb5c2ac85c85c54` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:52.961362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5895d8289ac644b4afb5c2ac85c85c54)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:52.972218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:53.002067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:53.073422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5895d8289ac644b4afb5c2ac85c85c54', 'request_id': 'req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.101359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5895d8289ac644b4afb5c2ac85c85c54', 'request_id': 'req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.111185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5895d8289ac644b4afb5c2ac85c85c54', 'request_id': 'req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.153887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] The token cache is being invalidate because project 5895d8289ac644b4afb5c2ac85c85c54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:53.157261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.158825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.165011 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 751/1471] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:52 2022] DELETE /identity/v3/projects/5895d8289ac644b4afb5c2ac85c85c54 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:55:53.165273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:53.186141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160698c7-2887-42bc-9c00-e537560f63ea service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:53.402844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:53.405245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:53.406878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:53.413262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] PATH_INFO: `/v3/users/8ca9787790e24b83bbeddfd35f61a6d2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:53.427996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca9787790e24b83bbeddfd35f61a6d2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:53.431293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:53.435764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:53.696136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca9787790e24b83bbeddfd35f61a6d2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.708028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca9787790e24b83bbeddfd35f61a6d2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.745141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca9787790e24b83bbeddfd35f61a6d2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.779017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca9787790e24b83bbeddfd35f61a6d2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.780025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca9787790e24b83bbeddfd35f61a6d2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:53.817654 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 721/1472] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:55:53 2022] DELETE /identity/v3/users/8ca9787790e24b83bbeddfd35f61a6d2 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:54.604009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:54.606499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:54.751146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c2787f-9b7e-406c-a2dc-4192e9a100fb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:54.946948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef34fd-dbe2-418c-a58f-1e3c456ef020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:54.949697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef34fd-dbe2-418c-a58f-1e3c456ef020 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:54.950018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef34fd-dbe2-418c-a58f-1e3c456ef020 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:54.950248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef34fd-dbe2-418c-a58f-1e3c456ef020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:54.954656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bbef34fd-dbe2-418c-a58f-1e3c456ef020 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:55.154111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef34fd-dbe2-418c-a58f-1e3c456ef020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:55.155582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef34fd-dbe2-418c-a58f-1e3c456ef020 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:55.170652 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 752/1473] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:55:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:55:57.198868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:57.200754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:57.438392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90280f9-96c3-46bd-860b-bffc9926ca3b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:57.446891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:57.448016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:57.448874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:57.449446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] PATH_INFO: `/v3/projects/c03a04af6f6b4fcf90525d89bb327974` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:57.451638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c03a04af6f6b4fcf90525d89bb327974)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:57.454571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:57.558877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c03a04af6f6b4fcf90525d89bb327974', 'request_id': 'req-5c36d58d-1c28-4211-9350-9c5e67eaa401'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:57.581415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c03a04af6f6b4fcf90525d89bb327974', 'request_id': 'req-5c36d58d-1c28-4211-9350-9c5e67eaa401'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:57.589106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c03a04af6f6b4fcf90525d89bb327974', 'request_id': 'req-5c36d58d-1c28-4211-9350-9c5e67eaa401'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:57.617243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] The token cache is being invalidate because project c03a04af6f6b4fcf90525d89bb327974 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:55:57.617587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:57.627343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c36d58d-1c28-4211-9350-9c5e67eaa401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:55:57.634769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48e5f-a531-4663-b5b7-033e59efa51c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:57.635377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48e5f-a531-4663-b5b7-033e59efa51c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:57.635377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff48e5f-a531-4663-b5b7-033e59efa51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:57.637115 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 722/1474] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:55:57 2022] DELETE /identity/v3/projects/c03a04af6f6b4fcf90525d89bb327974 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:57.678613 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1ff48e5f-a531-4663-b5b7-033e59efa51c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:57.679993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1ff48e5f-a531-4663-b5b7-033e59efa51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:57.694763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:57.913641 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 753/1475] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:57.915151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc44660-1f4c-4ac0-a86b-f709b67cdd53 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:57.915387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc44660-1f4c-4ac0-a86b-f709b67cdd53 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:57.915758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc44660-1f4c-4ac0-a86b-f709b67cdd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:57.957155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9dc44660-1f4c-4ac0-a86b-f709b67cdd53 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:57.958485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9dc44660-1f4c-4ac0-a86b-f709b67cdd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:58.000486 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 723/1476] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:58.006303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff48e5f-a531-4663-b5b7-033e59efa51c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:58.032973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff48e5f-a531-4663-b5b7-033e59efa51c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:58.061015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4547a027-6ad1-4489-b7fd-6a68af604021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:58.061666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a027-6ad1-4489-b7fd-6a68af604021 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:58.061945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a027-6ad1-4489-b7fd-6a68af604021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:58.062257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4547a027-6ad1-4489-b7fd-6a68af604021 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:58.063006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547a027-6ad1-4489-b7fd-6a68af604021 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:55:58.063326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547a027-6ad1-4489-b7fd-6a68af604021 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:58.070543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4547a027-6ad1-4489-b7fd-6a68af604021 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:58.118851 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 754/1477] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:55:58.119772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc44660-1f4c-4ac0-a86b-f709b67cdd53 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:58.128443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:58.139672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc44660-1f4c-4ac0-a86b-f709b67cdd53 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:58.166313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20ebbe8-2d90-47e6-bacf-faca958830a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:58.168756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ebbe8-2d90-47e6-bacf-faca958830a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:58.169959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ebbe8-2d90-47e6-bacf-faca958830a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:58.172965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20ebbe8-2d90-47e6-bacf-faca958830a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:58.177626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ebbe8-2d90-47e6-bacf-faca958830a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:58.181221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20ebbe8-2d90-47e6-bacf-faca958830a0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:58.214843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7739c-f79a-45c5-8773-da16caee2463 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:58.215507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7739c-f79a-45c5-8773-da16caee2463 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:58.215964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7739c-f79a-45c5-8773-da16caee2463 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:58.217925 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 755/1478] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:55:58.271367 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 724/1479] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:58 2022] POST /identity/v3/projects => generated 387 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:55:58.272795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7739c-f79a-45c5-8773-da16caee2463 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:58.475973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede709-e65c-4497-a4ec-9ed50a8e90aa None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:58.476402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede709-e65c-4497-a4ec-9ed50a8e90aa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:58.476776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ede709-e65c-4497-a4ec-9ed50a8e90aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:58.517785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-17ede709-e65c-4497-a4ec-9ed50a8e90aa None None] MFA Rules not processed for user `959991a17c69459dbd665c2ae20095ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:58.520628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-17ede709-e65c-4497-a4ec-9ed50a8e90aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:58.532804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7739c-f79a-45c5-8773-da16caee2463 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:58.543715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5d212-7b2c-4383-857f-5ec3a9f038b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:58.544382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d212-7b2c-4383-857f-5ec3a9f038b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:58.544666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d212-7b2c-4383-857f-5ec3a9f038b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:58.544939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5d212-7b2c-4383-857f-5ec3a9f038b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:58.545689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5d212-7b2c-4383-857f-5ec3a9f038b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:58.559674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f5d212-7b2c-4383-857f-5ec3a9f038b9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:58.662538 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 756/1480] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:58 2022] POST /identity/v3/users => generated 294 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:55:58.666261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cb339-92a8-4485-bc08-9daccdddc447 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:58.669911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cb339-92a8-4485-bc08-9daccdddc447 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:58.670196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cb339-92a8-4485-bc08-9daccdddc447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:58.760982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4b3cb339-92a8-4485-bc08-9daccdddc447 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:58.762626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4b3cb339-92a8-4485-bc08-9daccdddc447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:58.994097 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 757/1481] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:55:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:58.997259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cb339-92a8-4485-bc08-9daccdddc447 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:59.030124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cb339-92a8-4485-bc08-9daccdddc447 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:59.045894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:59.046645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:59.052182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:59.052828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:59.053556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:59.059308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:59.100681 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 758/1482] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:59.102577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:59.136016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e9f64d-934a-4420-bf68-e40d6a3d32b9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:59.153661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b4a43-19c0-4d73-bafc-40d57c6c0eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:59.154295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4a43-19c0-4d73-bafc-40d57c6c0eca admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:59.155981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4a43-19c0-4d73-bafc-40d57c6c0eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:59.156892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b4a43-19c0-4d73-bafc-40d57c6c0eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:59.164196 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 725/1483] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 690 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:55:59.171880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b4a43-19c0-4d73-bafc-40d57c6c0eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:59.174079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4b4a43-19c0-4d73-bafc-40d57c6c0eca admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:59.195886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:55:59.215450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede709-e65c-4497-a4ec-9ed50a8e90aa None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:59.216569 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 759/1484] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:55:59.249623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ede709-e65c-4497-a4ec-9ed50a8e90aa None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:59.264571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:59.265168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:59.265349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:59.265640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] PATH_INFO: `/v3/projects/b4e18562eadd4e7388fe7ba26e0d12ad/users/fd5ab2e406cd4eff8ecfa8e9a5973620/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:59.309229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4e18562eadd4e7388fe7ba26e0d12ad, user_id=fd5ab2e406cd4eff8ecfa8e9a5973620, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:59.315856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:59.344799 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 726/1485] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:55:59 2022] PUT /identity/v3/projects/b4e18562eadd4e7388fe7ba26e0d12ad/users/fd5ab2e406cd4eff8ecfa8e9a5973620/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:55:59.358333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90558ad5-a542-43cf-a256-86712edbfd2d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:59.358490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90558ad5-a542-43cf-a256-86712edbfd2d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:59.358719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90558ad5-a542-43cf-a256-86712edbfd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:59.457962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-90558ad5-a542-43cf-a256-86712edbfd2d None None] MFA Rules not processed for user `fd5ab2e406cd4eff8ecfa8e9a5973620`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:55:59.459225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-90558ad5-a542-43cf-a256-86712edbfd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:55:59.603609 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 760/1486] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:55:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:55:59.604345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:59.798804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324b85e-bf67-4963-935a-326da0caf6b2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:55:59.821407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:55:59.821980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:55:59.822469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:55:59.822798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:55:59.823546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:55:59.833613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:55:59.892326 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 727/1487] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:55:59 2022] POST /identity/v3/projects => generated 387 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:55:59.896058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90558ad5-a542-43cf-a256-86712edbfd2d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:55:59.917155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90558ad5-a542-43cf-a256-86712edbfd2d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:00.014337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:00.020399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:00.026810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:00.027681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.028007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.028389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.033647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:00.036137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:00.112835 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 761/1488] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:55:59 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:00.113852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:00.124739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:00.158823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ce7e0-1dab-4d8c-90e4-006532691bb6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:00.171139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5f961-5688-488c-98bd-acce63f45678 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:00.182295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:00.182972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.183381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.183722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.184965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:00.185974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:00.189911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7b6c6-415e-4878-8a20-5cff3f15ca9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:00.190546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b6c6-415e-4878-8a20-5cff3f15ca9d service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.190858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b6c6-415e-4878-8a20-5cff3f15ca9d service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.191178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7b6c6-415e-4878-8a20-5cff3f15ca9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.202654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b6c6-415e-4878-8a20-5cff3f15ca9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:00.205168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7b6c6-415e-4878-8a20-5cff3f15ca9d service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:00.220778 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 728/1489] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:00.222287 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 762/1490] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:00.222842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b8e7-3481-4c10-aa00-f1217239dcfb None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.223371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b8e7-3481-4c10-aa00-f1217239dcfb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.223977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790b8e7-3481-4c10-aa00-f1217239dcfb None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.225631 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 729/1491] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:56:00.226564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:00.248858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ceb153-59da-4a65-92e2-7aa310d144a5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:00.262061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:00.262790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.263073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.263391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] PATH_INFO: `/v3/projects/bed827a488954e5e8dc2a496de64f8ed/users/fd4090b05b5a413b92f1cc55e50800e8/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.307726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed827a488954e5e8dc2a496de64f8ed, user_id=fd4090b05b5a413b92f1cc55e50800e8, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:00.317742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:00.324288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.324753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.324903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.363752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:00.365039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:00.379266 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 763/1492] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:00 2022] PUT /identity/v3/projects/bed827a488954e5e8dc2a496de64f8ed/users/fd4090b05b5a413b92f1cc55e50800e8/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:00.383743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:00.544491 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 730/1493] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:00.546485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:00.563460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:00.566345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:00.602142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fabcb65-50ff-404a-a194-8339b54a703e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:00.618139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:00.620596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.620596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.620818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.626946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:00.645285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:00.675190 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 764/1494] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:00.684073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:00.736984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e67d5dd-31e0-4004-b00d-31d78d2f5205 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:00.751216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc7773-d97e-4b16-95b0-ceb07251225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:00.753860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7773-d97e-4b16-95b0-ceb07251225c admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.755881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7773-d97e-4b16-95b0-ceb07251225c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.757847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7773-d97e-4b16-95b0-ceb07251225c admin admin] PATH_INFO: `/v3/projects/bed827a488954e5e8dc2a496de64f8ed/users/fd4090b05b5a413b92f1cc55e50800e8/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.761724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc7773-d97e-4b16-95b0-ceb07251225c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bed827a488954e5e8dc2a496de64f8ed, user_id=fd4090b05b5a413b92f1cc55e50800e8, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:00.767097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5e069-725c-4174-b8dc-5c0344170714 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:00.769593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acc7773-d97e-4b16-95b0-ceb07251225c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:00.790830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:00.791494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.791782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.792054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] PATH_INFO: `/v3/users/776a1fde3f9e47bf8ca74f75e6693f19` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.795284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=776a1fde3f9e47bf8ca74f75e6693f19)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:00.805795 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 765/1495] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:00 2022] PUT /identity/v3/projects/bed827a488954e5e8dc2a496de64f8ed/users/fd4090b05b5a413b92f1cc55e50800e8/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:00.806011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:00.808460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.808816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.809154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:00.864678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] MFA Rules not processed for user `fd4090b05b5a413b92f1cc55e50800e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:00.866196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:00.894822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '776a1fde3f9e47bf8ca74f75e6693f19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:00.895437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '776a1fde3f9e47bf8ca74f75e6693f19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:00.905445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '776a1fde3f9e47bf8ca74f75e6693f19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:00.915281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:00.944539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '776a1fde3f9e47bf8ca74f75e6693f19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:00.946285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-131bc612-4b82-498c-a3a6-9600203f2017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '776a1fde3f9e47bf8ca74f75e6693f19'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:00.977528 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 731/1496] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:00 2022] DELETE /identity/v3/users/776a1fde3f9e47bf8ca74f75e6693f19 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:00.982846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:00.983134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:00.983459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:01.022506 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 766/1497] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:01.035993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:01.039845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:01.042094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:01.043263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:01.178449 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 732/1498] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:01.214105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:01.215923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:01.222196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fcf815-fdff-431a-89b0-0f063a53207d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:01.326368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:01.327004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:01.327283 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:01.327559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:01.371110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:01.453954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d43d0-6f49-4895-8151-51d00b781d64 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:01.458497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:01.460080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:01.468148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:01.472166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:01.472736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:01.473252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:01.486685 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 767/1499] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:01 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:01.495339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:01.496973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:01.517566 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 733/1500] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:02.199833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:02.202262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:02.233042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7123f64-1428-402d-b09f-328b74dbf8c8 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:02.248606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:02.249729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:02.250301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:02.250756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] PATH_INFO: `/v3/projects/6b0df7794db74f469bd8e7e52a5ceb67` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:02.252441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b0df7794db74f469bd8e7e52a5ceb67)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:02.260115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:02.370535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b0df7794db74f469bd8e7e52a5ceb67', 'request_id': 'req-586c8acb-c475-4738-83b6-bfb31d8128ac'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.382056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b0df7794db74f469bd8e7e52a5ceb67', 'request_id': 'req-586c8acb-c475-4738-83b6-bfb31d8128ac'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.393062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b0df7794db74f469bd8e7e52a5ceb67', 'request_id': 'req-586c8acb-c475-4738-83b6-bfb31d8128ac'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.412379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] The token cache is being invalidate because project 6b0df7794db74f469bd8e7e52a5ceb67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:02.413420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.414391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.417587 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 768/1501] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:02 2022] DELETE /identity/v3/projects/6b0df7794db74f469bd8e7e52a5ceb67 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:02.421574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:02.435671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad4e86-170e-4887-bcc3-f92e506d162c service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:02.565378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:02.567772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:02.569833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:02.570582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:02.570857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:02.571516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] PATH_INFO: `/v3/users/804efaafbc3e47bfae1e6d193ccde3a3` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:02.572712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=804efaafbc3e47bfae1e6d193ccde3a3)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:02.574787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:02.583513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586c8acb-c475-4738-83b6-bfb31d8128ac admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:02.589941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:02.592844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:02.593282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:02.595439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] PATH_INFO: `/v3/users/fd5ab2e406cd4eff8ecfa8e9a5973620` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:02.598482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5ab2e406cd4eff8ecfa8e9a5973620)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:02.605266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:02.619712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:02.639242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '804efaafbc3e47bfae1e6d193ccde3a3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.641717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '804efaafbc3e47bfae1e6d193ccde3a3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.650142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '804efaafbc3e47bfae1e6d193ccde3a3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.665098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '804efaafbc3e47bfae1e6d193ccde3a3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.665925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '804efaafbc3e47bfae1e6d193ccde3a3'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.669548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5ab2e406cd4eff8ecfa8e9a5973620'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.678754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5ab2e406cd4eff8ecfa8e9a5973620'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.681309 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 734/1502] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:02 2022] DELETE /identity/v3/users/804efaafbc3e47bfae1e6d193ccde3a3 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:02.682019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:02.683437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:02.694539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5ab2e406cd4eff8ecfa8e9a5973620'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.715218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5ab2e406cd4eff8ecfa8e9a5973620'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.716874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e45f1eca-29da-4080-ba81-71dc9386a50d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5ab2e406cd4eff8ecfa8e9a5973620'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:02.726649 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 769/1503] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:02 2022] DELETE /identity/v3/users/fd5ab2e406cd4eff8ecfa8e9a5973620 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:02.731356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:02.731768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:02.732782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:02.767310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:02.769060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:02.777227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:02.782191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847d7b3-6e30-4526-b9f4-8bacd7bdb78b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:02.868526 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 770/1504] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:02.885177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:02.885982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:02.886817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:02.887136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:02.887721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:02.888324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:02.890725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:02.906769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:02.909421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:02.928113 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 735/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:03.028750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a95264-6675-4627-84fb-56f5899ec6f1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.040224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99847307-f40c-478e-9d7d-66ac5cd36635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.040996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847307-f40c-478e-9d7d-66ac5cd36635 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.041274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847307-f40c-478e-9d7d-66ac5cd36635 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.041553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99847307-f40c-478e-9d7d-66ac5cd36635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.053086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847307-f40c-478e-9d7d-66ac5cd36635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.054595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99847307-f40c-478e-9d7d-66ac5cd36635 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.065960 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 771/1506] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:03.066344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.067913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:03.087519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55f4e4-872a-462e-9014-48a59d6fd9e6 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.093194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.094314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.094758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.095277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] PATH_INFO: `/v3/projects/a07e4adadc0e4c42a32b6653abe8d201` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.096889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a07e4adadc0e4c42a32b6653abe8d201)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.099240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.142119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:03.169430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a07e4adadc0e4c42a32b6653abe8d201', 'request_id': 'req-c62a3408-dbf9-4c49-a6ad-efaa570281da'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:03.182419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a07e4adadc0e4c42a32b6653abe8d201', 'request_id': 'req-c62a3408-dbf9-4c49-a6ad-efaa570281da'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:03.192545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a07e4adadc0e4c42a32b6653abe8d201', 'request_id': 'req-c62a3408-dbf9-4c49-a6ad-efaa570281da'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:03.207302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] The token cache is being invalidate because project a07e4adadc0e4c42a32b6653abe8d201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:03.207690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:03.208688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c62a3408-dbf9-4c49-a6ad-efaa570281da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:03.210653 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 736/1507] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:03 2022] DELETE /identity/v3/projects/a07e4adadc0e4c42a32b6653abe8d201 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:03.215558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502e6ea-7e45-4403-8fac-ea7abce8c1ba None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.216048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502e6ea-7e45-4403-8fac-ea7abce8c1ba None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.216048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502e6ea-7e45-4403-8fac-ea7abce8c1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.248384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f502e6ea-7e45-4403-8fac-ea7abce8c1ba None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:03.249721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f502e6ea-7e45-4403-8fac-ea7abce8c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:03.362463 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 772/1508] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:03.372374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7bf1-54ae-4122-8e71-15cc1b952571 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.372662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7bf1-54ae-4122-8e71-15cc1b952571 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.372938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95e7bf1-54ae-4122-8e71-15cc1b952571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.394189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d95e7bf1-54ae-4122-8e71-15cc1b952571 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:03.395453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d95e7bf1-54ae-4122-8e71-15cc1b952571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:03.409629 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 737/1509] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:03.413644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502e6ea-7e45-4403-8fac-ea7abce8c1ba None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.426698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502e6ea-7e45-4403-8fac-ea7abce8c1ba None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.433406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.433672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.434006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.434364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.435129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:03.435647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.437377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.451937 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 773/1510] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:03.456607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e7bf1-54ae-4122-8e71-15cc1b952571 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.469199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95e7bf1-54ae-4122-8e71-15cc1b952571 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.481513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.482358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.482724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.483094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.483834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.486421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.528077 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 738/1511] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:03 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:03.529638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.549440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feeb23d-0a45-4b78-a1e6-b147f1c0e684 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.630074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:03.680589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.682618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.684182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.684599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.685725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.687972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.728533 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 774/1512] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:03 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:03.730883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.754369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030d403-e0b9-4622-8439-d97b669eaae9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.777305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.778253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.778253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.778690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.779447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.781265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.807940 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 739/1513] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:03.809348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.824508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082f3390-878b-4ebd-b8bb-8672c0f4e846 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.829779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.830469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.830830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.831297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.832014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.833793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.856792 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 775/1514] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:03.861817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.877330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdac61-6ced-4ea8-9551-9b87bb1f056e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.884779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164de9bf-59ca-49ce-9a6c-7b2c9dc2794a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:03.885662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164de9bf-59ca-49ce-9a6c-7b2c9dc2794a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.885942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164de9bf-59ca-49ce-9a6c-7b2c9dc2794a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.886210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164de9bf-59ca-49ce-9a6c-7b2c9dc2794a admin admin] PATH_INFO: `/v3/projects/d18ee4994eb84e31b1480617f80cd2d7/users/97e9a3f5f5c7462e95c9f1150f5bda77/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.904350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164de9bf-59ca-49ce-9a6c-7b2c9dc2794a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18ee4994eb84e31b1480617f80cd2d7, user_id=97e9a3f5f5c7462e95c9f1150f5bda77, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:03.907759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164de9bf-59ca-49ce-9a6c-7b2c9dc2794a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:03.922409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:03.924122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:03.933427 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 740/1515] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:03 2022] PUT /identity/v3/projects/d18ee4994eb84e31b1480617f80cd2d7/users/97e9a3f5f5c7462e95c9f1150f5bda77/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:03.935876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858e1e1-f225-4738-9202-0b4c80dfa3f5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:03.936260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858e1e1-f225-4738-9202-0b4c80dfa3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:03.936948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858e1e1-f225-4738-9202-0b4c80dfa3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:03.945267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cfba4b-e660-4655-8c07-3958b367c5da admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:03.969842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9858e1e1-f225-4738-9202-0b4c80dfa3f5 None None] MFA Rules not processed for user `97e9a3f5f5c7462e95c9f1150f5bda77`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:03.971075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9858e1e1-f225-4738-9202-0b4c80dfa3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:04.008545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.009186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.009467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.009746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] PATH_INFO: `/v3/projects/b4e18562eadd4e7388fe7ba26e0d12ad` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.010868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4e18562eadd4e7388fe7ba26e0d12ad)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.012861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.040153 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 741/1516] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:04.044834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858e1e1-f225-4738-9202-0b4c80dfa3f5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.055184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858e1e1-f225-4738-9202-0b4c80dfa3f5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:04.061727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4e18562eadd4e7388fe7ba26e0d12ad', 'request_id': 'req-be42124d-f288-4668-adbe-3752210abf37'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.070498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4e18562eadd4e7388fe7ba26e0d12ad', 'request_id': 'req-be42124d-f288-4668-adbe-3752210abf37'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.077292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4e18562eadd4e7388fe7ba26e0d12ad', 'request_id': 'req-be42124d-f288-4668-adbe-3752210abf37'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.088597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] The token cache is being invalidate because project b4e18562eadd4e7388fe7ba26e0d12ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:04.088886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.089594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.091097 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 776/1517] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:03 2022] DELETE /identity/v3/projects/b4e18562eadd4e7388fe7ba26e0d12ad => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:04.095517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.097064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:04.097221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:04.118407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.119101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.119383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.119655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.120283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.122205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.149614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:04.152563 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 742/1518] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:04 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:04.155733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.158006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:04.173880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42124d-f288-4668-adbe-3752210abf37 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:04.230737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603af14f-2f5a-4dc7-8155-8da708eeb9ae admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:04.242242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.242866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.243144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.243383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.243838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.244032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.244126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.244374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.244647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] PATH_INFO: `/v3/users/fd4090b05b5a413b92f1cc55e50800e8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.245790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd4090b05b5a413b92f1cc55e50800e8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.245969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.247861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.272923 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 743/1519] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:04 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:04.276743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.288732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7f08c-9b1b-4a75-8e74-e9f656684164 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:04.294857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.295567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.295951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.296360 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.297476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.308111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.320897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd4090b05b5a413b92f1cc55e50800e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.334836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd4090b05b5a413b92f1cc55e50800e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.341914 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 744/1520] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:04.343466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.352155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd4090b05b5a413b92f1cc55e50800e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.371089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd4090b05b5a413b92f1cc55e50800e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.372739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd4090b05b5a413b92f1cc55e50800e8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:04.376114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa2e9e-b91d-4a0a-a5d1-1a1f89c5d8a8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:04.383914 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 777/1521] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:04 2022] DELETE /identity/v3/users/fd4090b05b5a413b92f1cc55e50800e8 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:04.389716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.390541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.390905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.391820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] PATH_INFO: `/v3/projects/32979c839a884bf9883e21bf14fb4266/users/2609996f15bf413ebc44e8921791232f/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.437220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32979c839a884bf9883e21bf14fb4266, user_id=2609996f15bf413ebc44e8921791232f, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.443522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.472355 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 745/1522] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:04 2022] PUT /identity/v3/projects/32979c839a884bf9883e21bf14fb4266/users/2609996f15bf413ebc44e8921791232f/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:04.475144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.476913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:04.596721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fe6a2-7c84-417f-bb7d-c0ff229d4b10 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:04.650151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:04.749391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4abe42-1691-45f6-bf18-68a38970f652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.750166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4abe42-1691-45f6-bf18-68a38970f652 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.751054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4abe42-1691-45f6-bf18-68a38970f652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.751549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4abe42-1691-45f6-bf18-68a38970f652 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.752467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4abe42-1691-45f6-bf18-68a38970f652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.754556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4abe42-1691-45f6-bf18-68a38970f652 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.767433 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 778/1523] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:04.771269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.782675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58495e5-c45f-4247-a2a8-b1154265da37 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:04.788998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:04.789965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.790455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.790944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] PATH_INFO: `/v3/projects/32979c839a884bf9883e21bf14fb4266/users/2609996f15bf413ebc44e8921791232f/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.793884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32979c839a884bf9883e21bf14fb4266, user_id=2609996f15bf413ebc44e8921791232f, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:04.796776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:04.810358 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 746/1524] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:04 2022] PUT /identity/v3/projects/32979c839a884bf9883e21bf14fb4266/users/2609996f15bf413ebc44e8921791232f/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:04.822736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:04.824323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:04.824323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:04.859321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] MFA Rules not processed for user `2609996f15bf413ebc44e8921791232f`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:04.861525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:04.975429 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 779/1525] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:04.988119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:04.989842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:05.054421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3e2461-5f1a-4fe6-9380-b14e1128a087 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:05.058103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca68a1b-d9db-4250-9c25-0c4188eb186d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:05.058800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca68a1b-d9db-4250-9c25-0c4188eb186d service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.059005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca68a1b-d9db-4250-9c25-0c4188eb186d service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.059233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca68a1b-d9db-4250-9c25-0c4188eb186d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.060970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8ca68a1b-d9db-4250-9c25-0c4188eb186d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:05.081460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:05.161254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca68a1b-d9db-4250-9c25-0c4188eb186d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:05.163060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca68a1b-d9db-4250-9c25-0c4188eb186d service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:05.180525 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 747/1526] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:05.358195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:05.359931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:05.482859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6dce98-d783-47da-830b-cc993d6136fd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:05.489379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:05.490163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.490564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.490943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] PATH_INFO: `/v3/projects/bed827a488954e5e8dc2a496de64f8ed` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.492190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bed827a488954e5e8dc2a496de64f8ed)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:05.494510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:05.562045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:05.564182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bed827a488954e5e8dc2a496de64f8ed', 'request_id': 'req-7db27d79-1a2b-4b10-994d-c1c1c26bc807'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:05.575547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bed827a488954e5e8dc2a496de64f8ed', 'request_id': 'req-7db27d79-1a2b-4b10-994d-c1c1c26bc807'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:05.585158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bed827a488954e5e8dc2a496de64f8ed', 'request_id': 'req-7db27d79-1a2b-4b10-994d-c1c1c26bc807'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:05.602001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] The token cache is being invalidate because project bed827a488954e5e8dc2a496de64f8ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:05.602420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:05.603315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7db27d79-1a2b-4b10-994d-c1c1c26bc807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:05.605468 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 780/1527] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:05 2022] DELETE /identity/v3/projects/bed827a488954e5e8dc2a496de64f8ed => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:05.611058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f530a-2574-453f-a8a1-e8a5de8fda9b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.611283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f530a-2574-453f-a8a1-e8a5de8fda9b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.611560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f530a-2574-453f-a8a1-e8a5de8fda9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.632628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ad3f530a-2574-453f-a8a1-e8a5de8fda9b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:05.634067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ad3f530a-2574-453f-a8a1-e8a5de8fda9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:05.711858 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 748/1528] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:05.720168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187a00-64fc-4832-98bc-7bc21ecb8822 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.720557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187a00-64fc-4832-98bc-7bc21ecb8822 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.721152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23187a00-64fc-4832-98bc-7bc21ecb8822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.744004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-23187a00-64fc-4832-98bc-7bc21ecb8822 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:05.744942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-23187a00-64fc-4832-98bc-7bc21ecb8822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:05.754955 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 781/1529] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:05.759263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f530a-2574-453f-a8a1-e8a5de8fda9b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:05.768932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f530a-2574-453f-a8a1-e8a5de8fda9b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:05.773528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:05.774206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.774491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.774763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.775403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:05.775701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:05.777293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:05.790019 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 749/1530] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:05.794506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23187a00-64fc-4832-98bc-7bc21ecb8822 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:05.804619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23187a00-64fc-4832-98bc-7bc21ecb8822 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:05.809529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:05.810444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.810866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.811281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.812067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:05.814468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:05.847954 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 782/1531] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:05 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:05.849093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:05.860125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9219c7d-0d59-4449-b1eb-6e280a69d625 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:05.919533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:05.920143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.920459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.920732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.921355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:05.923851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:05.945335 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 750/1532] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:05 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:05.949760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:05.960377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b8f98-696a-4be6-920d-4c71a785194b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:05.965813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:05.966427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:05.966698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:05.966962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:05.967598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:05.970124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:05.983483 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 783/1533] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:05.985306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:05.997989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51111091-5c5d-41b0-b282-c58e9586ec7e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.002738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf9993-9b85-4455-a378-454841835261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.003650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf9993-9b85-4455-a378-454841835261 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.003930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf9993-9b85-4455-a378-454841835261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.004482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf9993-9b85-4455-a378-454841835261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.005127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf9993-9b85-4455-a378-454841835261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.006696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf9993-9b85-4455-a378-454841835261 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.018561 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 751/1534] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:06.022797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.034100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9901070-d6c4-4cca-a757-b27b830825f2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.039085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.039745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.040027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.040322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] PATH_INFO: `/v3/projects/49b4fe211bf14dd99ce83f5deac0fbd1/users/97deceebfa70472a9662a892e26ad2e8/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.055572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49b4fe211bf14dd99ce83f5deac0fbd1, user_id=97deceebfa70472a9662a892e26ad2e8, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.058256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.070558 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 784/1535] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:06 2022] PUT /identity/v3/projects/49b4fe211bf14dd99ce83f5deac0fbd1/users/97deceebfa70472a9662a892e26ad2e8/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:06.080980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f03fc-8a5f-4325-94dd-4b17cef3f1b7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.081261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f03fc-8a5f-4325-94dd-4b17cef3f1b7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.081525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f03fc-8a5f-4325-94dd-4b17cef3f1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.110255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bf2f03fc-8a5f-4325-94dd-4b17cef3f1b7 None None] MFA Rules not processed for user `97deceebfa70472a9662a892e26ad2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:06.111603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bf2f03fc-8a5f-4325-94dd-4b17cef3f1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:06.124606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:06.179102 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 752/1536] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:06.183673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.253792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76379e6-c1ed-40ed-b0e2-58ba96b53ce4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.258462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.259068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.259350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.259620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.260285 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.262202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.291509 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 785/1537] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:06 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:06.295490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f03fc-8a5f-4325-94dd-4b17cef3f1b7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.307085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f03fc-8a5f-4325-94dd-4b17cef3f1b7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.369349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.369975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.370259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.370535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.371173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.373174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.406018 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 753/1538] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:06 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:06.410489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.422201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349a57f-1fe5-41b5-a2ab-35306795cf65 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.428076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.428792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.429074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.429386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.429978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.432194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.447655 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 786/1539] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:06.449535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.460751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc54f6c-a77f-4be8-b35a-74cd4bfcf07e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.465552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.466220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.466498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.466767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] PATH_INFO: `/v3/projects/12f2d6a45a5e487582dc2fe3c70bb921/users/b79df52247174b4b941ab2d8e1f4104a/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.488174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f2d6a45a5e487582dc2fe3c70bb921, user_id=b79df52247174b4b941ab2d8e1f4104a, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.497085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.516449 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 754/1540] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:06 2022] PUT /identity/v3/projects/12f2d6a45a5e487582dc2fe3c70bb921/users/b79df52247174b4b941ab2d8e1f4104a/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:06.519832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.530385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4f1d2-dded-4c04-a339-a9b885a92b8d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.586958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:06.602434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a470e9d8-859f-4321-8ea2-0edc3929fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.603073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e9d8-859f-4321-8ea2-0edc3929fb1a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.603482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e9d8-859f-4321-8ea2-0edc3929fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.603762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a470e9d8-859f-4321-8ea2-0edc3929fb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.604429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470e9d8-859f-4321-8ea2-0edc3929fb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.606008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a470e9d8-859f-4321-8ea2-0edc3929fb1a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.617859 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 787/1541] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:06 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:06.622938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.640910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17ef92-a05f-4aa9-8737-cd39936fba04 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.645885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.647647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.647934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.648197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] PATH_INFO: `/v3/projects/12f2d6a45a5e487582dc2fe3c70bb921/users/b79df52247174b4b941ab2d8e1f4104a/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.650140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12f2d6a45a5e487582dc2fe3c70bb921, user_id=b79df52247174b4b941ab2d8e1f4104a, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.653475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.668465 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 755/1542] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:06 2022] PUT /identity/v3/projects/12f2d6a45a5e487582dc2fe3c70bb921/users/b79df52247174b4b941ab2d8e1f4104a/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:06.677726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.678013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.678340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.707014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] MFA Rules not processed for user `b79df52247174b4b941ab2d8e1f4104a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:06.708216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:06.727817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.729605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:06.776514 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 788/1543] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:06.787208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a692eb-83f4-48f2-8bb5-417d98d6b242 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.789376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.790699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:06.791292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.791929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.792208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.792511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.794207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:06.803070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a694f9c3-86d9-4e27-831c-669f6763e6a3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.806666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.807262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.807542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.807809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:06.817951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.818390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:06.819542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.829465 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 789/1544] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:06.866130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:06.867888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:06.878131 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 756/1545] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:06 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:06.928826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:06.938549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4642490-a181-4ca0-9765-38fe6094123a service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:06.944061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02ae0d6-e664-47be-ad01-81f1675283f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:06.944775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ae0d6-e664-47be-ad01-81f1675283f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:06.946582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ae0d6-e664-47be-ad01-81f1675283f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:06.946929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02ae0d6-e664-47be-ad01-81f1675283f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:07.010736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ae0d6-e664-47be-ad01-81f1675283f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:07.012172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02ae0d6-e664-47be-ad01-81f1675283f2 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:07.023728 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 790/1546] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:07.313866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:07.379811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a3bdbd-df1b-4df9-aeb6-5808db53579c service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:07.384717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:07.385418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:07.385659 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:07.385933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] PATH_INFO: `/v3/users/97deceebfa70472a9662a892e26ad2e8` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:07.387035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97deceebfa70472a9662a892e26ad2e8)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:07.389026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:07.441260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97deceebfa70472a9662a892e26ad2e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:07.441918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97deceebfa70472a9662a892e26ad2e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:07.449413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97deceebfa70472a9662a892e26ad2e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:07.463648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97deceebfa70472a9662a892e26ad2e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:07.464468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97deceebfa70472a9662a892e26ad2e8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:07.477884 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 757/1547] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:07 2022] DELETE /identity/v3/users/97deceebfa70472a9662a892e26ad2e8 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:07.482140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:07.482639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:07.482728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:07.501603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:07.502793 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:07.552551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:07.565255 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 791/1548] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:07.574966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:07.576391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:07.623748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:07.649339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b731809-7b25-4ae1-8c94-3a969cfe8de5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:07.653831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:07.654823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:07.654823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:07.655260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:07.664643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:07.666065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:07.679562 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 758/1549] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:07.988641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:07.991257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:08.004920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402dcbf-282b-404a-92e4-d9b4c2baa3b7 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:08.010806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:08.011454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:08.011864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:08.012139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] PATH_INFO: `/v3/projects/49b4fe211bf14dd99ce83f5deac0fbd1` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:08.013660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49b4fe211bf14dd99ce83f5deac0fbd1)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:08.015954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:08.082439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49b4fe211bf14dd99ce83f5deac0fbd1', 'request_id': 'req-63897d02-ea8d-458b-b4b9-288620a139f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.095277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49b4fe211bf14dd99ce83f5deac0fbd1', 'request_id': 'req-63897d02-ea8d-458b-b4b9-288620a139f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.107233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49b4fe211bf14dd99ce83f5deac0fbd1', 'request_id': 'req-63897d02-ea8d-458b-b4b9-288620a139f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.120098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] The token cache is being invalidate because project 49b4fe211bf14dd99ce83f5deac0fbd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:08.124337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.128692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.130999 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 792/1550] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:07 2022] DELETE /identity/v3/projects/49b4fe211bf14dd99ce83f5deac0fbd1 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:08.135677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:08.149400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ba3a07-ebe4-4b4b-852b-48b784749415 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:08.252061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:08.253243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:08.253530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:08.253803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] PATH_INFO: `/v3/users/b79df52247174b4b941ab2d8e1f4104a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:08.263054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b79df52247174b4b941ab2d8e1f4104a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:08.265180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:08.313133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b79df52247174b4b941ab2d8e1f4104a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.313937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b79df52247174b4b941ab2d8e1f4104a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.321572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b79df52247174b4b941ab2d8e1f4104a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.334708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b79df52247174b4b941ab2d8e1f4104a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.335590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-136d82ee-c19a-4c4a-a0b6-23ee729483e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b79df52247174b4b941ab2d8e1f4104a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.349070 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 759/1551] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:08 2022] DELETE /identity/v3/users/b79df52247174b4b941ab2d8e1f4104a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:08.619948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:08.621423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:08.636227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63897d02-ea8d-458b-b4b9-288620a139f2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:08.712165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:08.712794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:08.713324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:08.713599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] PATH_INFO: `/v3/projects/12f2d6a45a5e487582dc2fe3c70bb921` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:08.714725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=12f2d6a45a5e487582dc2fe3c70bb921)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:08.716669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:08.769172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12f2d6a45a5e487582dc2fe3c70bb921', 'request_id': 'req-9508f03f-43c4-40bf-bab3-eca52f94ff7c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.777449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12f2d6a45a5e487582dc2fe3c70bb921', 'request_id': 'req-9508f03f-43c4-40bf-bab3-eca52f94ff7c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.784670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12f2d6a45a5e487582dc2fe3c70bb921', 'request_id': 'req-9508f03f-43c4-40bf-bab3-eca52f94ff7c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.798618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] The token cache is being invalidate because project 12f2d6a45a5e487582dc2fe3c70bb921 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:08.798997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.799700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9508f03f-43c4-40bf-bab3-eca52f94ff7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:08.801311 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 793/1552] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:08 2022] DELETE /identity/v3/projects/12f2d6a45a5e487582dc2fe3c70bb921 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:08.805995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757210e-1564-472d-8372-7e89163cb9af None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:08.806281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757210e-1564-472d-8372-7e89163cb9af None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:08.806555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757210e-1564-472d-8372-7e89163cb9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:08.829482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0757210e-1564-472d-8372-7e89163cb9af None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:08.830952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0757210e-1564-472d-8372-7e89163cb9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:08.909027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:08.924217 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 760/1553] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:08.933287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7b0a3-8ef8-4423-8a0d-e8e35676b172 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:08.933572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7b0a3-8ef8-4423-8a0d-e8e35676b172 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:08.933864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7b0a3-8ef8-4423-8a0d-e8e35676b172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:08.951508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:08.954876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e5d7b0a3-8ef8-4423-8a0d-e8e35676b172 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:08.956724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e5d7b0a3-8ef8-4423-8a0d-e8e35676b172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:08.976071 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 794/1554] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:08.979573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757210e-1564-472d-8372-7e89163cb9af None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:08.990197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0757210e-1564-472d-8372-7e89163cb9af None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:08.996353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:08.996353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:08.996353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:08.996353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:08.997156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:08.997524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:08.999187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.011715 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 761/1555] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:09.016614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7b0a3-8ef8-4423-8a0d-e8e35676b172 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.033668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7b0a3-8ef8-4423-8a0d-e8e35676b172 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.038566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.039129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.039655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.039907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.040654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.042878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.073351 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 795/1556] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:09 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:09.077486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.088143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aff599-8e39-4d8c-a85f-9af3e452e3e0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.147979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.148646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.148876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.149145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.149779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.151670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.174191 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 762/1557] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:09 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:09.178286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.189121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e7c59e-546d-4ccf-9bf2-3464727b6eb7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.193668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.194270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.194546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.194819 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.195460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.197094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.208988 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 796/1558] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:09.212992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.223515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a21b3-cedf-4a65-a0b5-f1a0ab30bd2f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.227988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ec697-ca36-4500-b9f5-144e58941d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.228578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ec697-ca36-4500-b9f5-144e58941d1e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.228850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ec697-ca36-4500-b9f5-144e58941d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.229119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ec697-ca36-4500-b9f5-144e58941d1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.229905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ec697-ca36-4500-b9f5-144e58941d1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.231477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ec697-ca36-4500-b9f5-144e58941d1e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.242842 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 763/1559] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:09.247215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.258647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b6ea3-fd8b-4a1e-8923-2ecdb15db912 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.266177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.266860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.267302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.267591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] PATH_INFO: `/v3/projects/d38a8c152f8d4d85b50912af027ff961/users/11ec83e5b20046e6aa48911663ce4a6b/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.286186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d38a8c152f8d4d85b50912af027ff961, user_id=11ec83e5b20046e6aa48911663ce4a6b, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.288928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.311906 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 797/1560] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:09 2022] PUT /identity/v3/projects/d38a8c152f8d4d85b50912af027ff961/users/11ec83e5b20046e6aa48911663ce4a6b/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:09.322591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5b04-9f9d-4607-aa4b-521725b30976 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.322926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5b04-9f9d-4607-aa4b-521725b30976 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.323230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5b04-9f9d-4607-aa4b-521725b30976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.362195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-21ef5b04-9f9d-4607-aa4b-521725b30976 None None] MFA Rules not processed for user `11ec83e5b20046e6aa48911663ce4a6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:09.363494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-21ef5b04-9f9d-4607-aa4b-521725b30976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:09.463690 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 764/1561] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:09.468637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.548463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e86d7c-5450-499e-93ce-881d2175993f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.557564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.558182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.558456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.558726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.559353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.561354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.588975 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 798/1562] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:09 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:09.592906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef5b04-9f9d-4607-aa4b-521725b30976 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.605610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef5b04-9f9d-4607-aa4b-521725b30976 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.658381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:09.667223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.668040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.668358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.668629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.669253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.671146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.692337 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 765/1563] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:09 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:09.696642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.707904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb28480-8732-4940-afc8-19cb842cdc19 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.714623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.715380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.716020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.716020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.716852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.719247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.731067 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 799/1564] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:09.736022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.746815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148842af-eb47-4394-9f43-d4e2b79c7dd2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.751663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9372ca-8e59-41e5-9291-ceffdd0b652a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.752273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9372ca-8e59-41e5-9291-ceffdd0b652a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.753094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9372ca-8e59-41e5-9291-ceffdd0b652a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.753216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9372ca-8e59-41e5-9291-ceffdd0b652a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.753895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9372ca-8e59-41e5-9291-ceffdd0b652a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.755620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9372ca-8e59-41e5-9291-ceffdd0b652a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.770360 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 766/1565] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:09.771694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:09.783537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b1f3d4-814d-43da-8b2c-6498926e35e4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:09.789002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:09.789659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.789949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.790216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] PATH_INFO: `/v3/projects/470454f897bb491aad98ed9a8f80e77d/users/204ad95a53e14c64bd4063e41e6daff4/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.806030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] RBAC: Authorizing `identity:create_grant(project_id=470454f897bb491aad98ed9a8f80e77d, user_id=204ad95a53e14c64bd4063e41e6daff4, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:09.814121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:09.827445 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 800/1566] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:09 2022] PUT /identity/v3/projects/470454f897bb491aad98ed9a8f80e77d/users/204ad95a53e14c64bd4063e41e6daff4/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:09.841513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeafa49-95e7-4b3e-9b66-473fd43815d1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:09.841800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeafa49-95e7-4b3e-9b66-473fd43815d1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:09.842088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeafa49-95e7-4b3e-9b66-473fd43815d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:09.871279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5eeafa49-95e7-4b3e-9b66-473fd43815d1 None None] MFA Rules not processed for user `204ad95a53e14c64bd4063e41e6daff4`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:09.872641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5eeafa49-95e7-4b3e-9b66-473fd43815d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:09.943153 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 767/1567] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:09.947882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.003298 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:10.015252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae9ff5-6292-4690-ab1a-a3a93355384c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.020266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.020931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.021216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.021489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.022138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.024252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.066388 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 801/1568] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:09 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:10.070566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeafa49-95e7-4b3e-9b66-473fd43815d1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.081375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeafa49-95e7-4b3e-9b66-473fd43815d1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.161522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.162205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.162545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.162878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.163568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.165660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.192051 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 768/1569] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:10 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:10.193302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.204392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4206343e-8184-44c7-abe5-7b2e814a2d46 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.209564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.210170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.210445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.210714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.211355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.212966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.233270 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 802/1570] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:10.235959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.248443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bf12df-526e-4046-bb1d-9c4e97e8b980 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.253982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.254769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.255146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.255520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] PATH_INFO: `/v3/projects/4db827b5ab61413584a71d6bf9d5ddc7/users/ffacc79000b5460c91e1b9957d144024/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.273913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db827b5ab61413584a71d6bf9d5ddc7, user_id=ffacc79000b5460c91e1b9957d144024, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.276645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.290188 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 769/1571] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:10 2022] PUT /identity/v3/projects/4db827b5ab61413584a71d6bf9d5ddc7/users/ffacc79000b5460c91e1b9957d144024/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:10.294275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.304807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaba483-1fa6-437d-afb7-6f0bc3bc94b8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.360440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a1173c-43ba-471f-9581-3b98d75af9a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.361043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1173c-43ba-471f-9581-3b98d75af9a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.361319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1173c-43ba-471f-9581-3b98d75af9a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.361603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a1173c-43ba-471f-9581-3b98d75af9a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.362220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1173c-43ba-471f-9581-3b98d75af9a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.363794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a1173c-43ba-471f-9581-3b98d75af9a5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.374387 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 803/1572] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:10.378790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.389110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a6795-575a-4f38-9d65-b55e8068e2dd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.393677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.394319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.394597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.394867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] PATH_INFO: `/v3/projects/4db827b5ab61413584a71d6bf9d5ddc7/users/ffacc79000b5460c91e1b9957d144024/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.396942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db827b5ab61413584a71d6bf9d5ddc7, user_id=ffacc79000b5460c91e1b9957d144024, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.399557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.410105 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 770/1573] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:10 2022] PUT /identity/v3/projects/4db827b5ab61413584a71d6bf9d5ddc7/users/ffacc79000b5460c91e1b9957d144024/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:10.419966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.420246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.420572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.447779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] MFA Rules not processed for user `ffacc79000b5460c91e1b9957d144024`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:10.449003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:10.467272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.469530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:10.488200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:10.520574 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 804/1574] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:10.532341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.534292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:10.536885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c46679-c6f8-4ec4-98b0-416f98f9d691 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.542199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.542828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.543103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.543372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] PATH_INFO: `/v3/users/df7a46c9bf1f47ca8fe22d7fb5a38e40` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.544517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df7a46c9bf1f47ca8fe22d7fb5a38e40)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.546406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.589632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:10.594873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f3ddb-a991-4fe4-a05a-f504cebaea7d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.600813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.602006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.602351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.602693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.605646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df7a46c9bf1f47ca8fe22d7fb5a38e40'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.606309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df7a46c9bf1f47ca8fe22d7fb5a38e40'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.616218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df7a46c9bf1f47ca8fe22d7fb5a38e40'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.630346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df7a46c9bf1f47ca8fe22d7fb5a38e40'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.631433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06fc30c5-0b04-4515-b236-8301fed2eb32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df7a46c9bf1f47ca8fe22d7fb5a38e40'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.646765 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 771/1575] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:10 2022] DELETE /identity/v3/users/df7a46c9bf1f47ca8fe22d7fb5a38e40 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:10.651017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.651454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.651752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.668087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.669883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.677005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:10.678206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:10.679778 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 805/1576] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:10 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:10.751507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.753277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:10.762112 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 772/1577] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:10.769448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c64fc3-8ead-45a0-9183-424902535ef6 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.770677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:10.772017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:10.774190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.775015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.775368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.775707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] PATH_INFO: `/v3/users/97e9a3f5f5c7462e95c9f1150f5bda77` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.776927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e9a3f5f5c7462e95c9f1150f5bda77)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.778921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.821819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:10.839373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e9a3f5f5c7462e95c9f1150f5bda77'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.849266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e9a3f5f5c7462e95c9f1150f5bda77'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.850474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe739d-dd34-4977-96fd-ce3ccec7ce38 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:10.855311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563dd879-c412-4de2-b054-5425f94d1b16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:10.855929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd879-c412-4de2-b054-5425f94d1b16 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.856264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd879-c412-4de2-b054-5425f94d1b16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.856710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563dd879-c412-4de2-b054-5425f94d1b16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.856896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e9a3f5f5c7462e95c9f1150f5bda77'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.866509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dd879-c412-4de2-b054-5425f94d1b16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:10.868159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563dd879-c412-4de2-b054-5425f94d1b16 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:10.875251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e9a3f5f5c7462e95c9f1150f5bda77'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.876184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e9a3f5f5c7462e95c9f1150f5bda77'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:10.878674 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 773/1578] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:10.885227 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 806/1579] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:10 2022] DELETE /identity/v3/users/97e9a3f5f5c7462e95c9f1150f5bda77 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:10.889842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:10.890227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:10.891098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:10.911432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:10.912750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:11.004335 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 774/1580] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:11.020667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:11.022602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:11.147043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6a730-d279-49c0-bc80-41a8f450483b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:11.151731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6fb8b1-911b-4d35-a9d0-8c5a1e6091b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:11.152533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb8b1-911b-4d35-a9d0-8c5a1e6091b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:11.153018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb8b1-911b-4d35-a9d0-8c5a1e6091b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:11.153416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6fb8b1-911b-4d35-a9d0-8c5a1e6091b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:11.175163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb8b1-911b-4d35-a9d0-8c5a1e6091b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:11.176965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6fb8b1-911b-4d35-a9d0-8c5a1e6091b9 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:11.198147 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 807/1581] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:11.341598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:11.344613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:11.357589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a242c58-6c5b-4e3d-b8bf-a518b064d909 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:11.363240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:11.363889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:11.364167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:11.364492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] PATH_INFO: `/v3/projects/d311f0fc85544325bb9400f0efd54b8a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:11.365621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d311f0fc85544325bb9400f0efd54b8a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:11.367531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:11.431470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd311f0fc85544325bb9400f0efd54b8a', 'request_id': 'req-e8b07738-e453-43c1-a8c2-072ceb462916'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.435457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1c87b-b16f-4b6c-8db3-0aa1f518f13b None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:11.435743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1c87b-b16f-4b6c-8db3-0aa1f518f13b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:11.436016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1c87b-b16f-4b6c-8db3-0aa1f518f13b None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:11.437501 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 808/1582] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:56:11.443065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd311f0fc85544325bb9400f0efd54b8a', 'request_id': 'req-e8b07738-e453-43c1-a8c2-072ceb462916'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.452684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd311f0fc85544325bb9400f0efd54b8a', 'request_id': 'req-e8b07738-e453-43c1-a8c2-072ceb462916'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.480556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:11.480556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:11.480556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:11.480889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] The token cache is being invalidate because project d311f0fc85544325bb9400f0efd54b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:11.482269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.483329 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.487009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 775/1583] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:11 2022] DELETE /identity/v3/projects/d311f0fc85544325bb9400f0efd54b8a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:11.490979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:11.492711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:11.507395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:11.509452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:11.526800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:11.655898 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 809/1584] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:56:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:11.658768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b07738-e453-43c1-a8c2-072ceb462916 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:11.663733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:11.664410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:11.664808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:11.665082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] PATH_INFO: `/v3/users/788e91ef45d94451bb4d20a530275e3a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:11.666217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=788e91ef45d94451bb4d20a530275e3a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:11.668128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:11.670666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:11.723123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '788e91ef45d94451bb4d20a530275e3a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.723905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '788e91ef45d94451bb4d20a530275e3a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.731395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '788e91ef45d94451bb4d20a530275e3a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.745394 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '788e91ef45d94451bb4d20a530275e3a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.746482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '788e91ef45d94451bb4d20a530275e3a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:11.770985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:11.772459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:11.782848 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 776/1585] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:11 2022] DELETE /identity/v3/users/788e91ef45d94451bb4d20a530275e3a => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:11.795802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6514e9-c405-4c3b-944f-13ab0f2995c8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:11.907293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:11.907930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:11.908210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:11.911513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:11.918387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:11.951876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:11.957094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:11.982425 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 810/1586] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:12.027437 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.028918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.144993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.146502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.158404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82cd745-91c7-444c-b4c2-cbf2b64f37a9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.175075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21215f44-17d0-42cc-8a4f-3ca8e0ff6e06 service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.175230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.176941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.177441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.178592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] PATH_INFO: `/v3/projects/d18ee4994eb84e31b1480617f80cd2d7` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.180123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18ee4994eb84e31b1480617f80cd2d7)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.183377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.183865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.184666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.184810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.185113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] PATH_INFO: `/v3/projects/a103b217545946d9860889815633d369` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.187124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a103b217545946d9860889815633d369)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.189625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.270076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a103b217545946d9860889815633d369', 'request_id': 'req-92c6de81-7c7b-4199-b484-3935d576457a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.278010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18ee4994eb84e31b1480617f80cd2d7', 'request_id': 'req-58c2f317-c701-4566-85c8-8073189bf170'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.283363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a103b217545946d9860889815633d369', 'request_id': 'req-92c6de81-7c7b-4199-b484-3935d576457a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.287514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18ee4994eb84e31b1480617f80cd2d7', 'request_id': 'req-58c2f317-c701-4566-85c8-8073189bf170'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.292104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a103b217545946d9860889815633d369', 'request_id': 'req-92c6de81-7c7b-4199-b484-3935d576457a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.294691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18ee4994eb84e31b1480617f80cd2d7', 'request_id': 'req-58c2f317-c701-4566-85c8-8073189bf170'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.304839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] The token cache is being invalidate because project a103b217545946d9860889815633d369 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:12.305027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.305676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.306636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] The token cache is being invalidate because project d18ee4994eb84e31b1480617f80cd2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:12.306922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.307184 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 811/1587] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:12 2022] DELETE /identity/v3/projects/a103b217545946d9860889815633d369 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:12.307806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-58c2f317-c701-4566-85c8-8073189bf170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.309380 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 777/1588] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:12 2022] DELETE /identity/v3/projects/d18ee4994eb84e31b1480617f80cd2d7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:12.314730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.315211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.315450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.315683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.316122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.335758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:12.336995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.340258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:12.344131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:12.407006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c6de81-7c7b-4199-b484-3935d576457a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.410560 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 778/1589] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:12.411406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.411644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.412281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.412589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.412867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] PATH_INFO: `/v3/users/2609996f15bf413ebc44e8921791232f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.414252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2609996f15bf413ebc44e8921791232f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.416183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.419630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.465347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2609996f15bf413ebc44e8921791232f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.472639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2609996f15bf413ebc44e8921791232f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.479357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58700-aa88-46dd-bab5-8f48d8549e8e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.480162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2609996f15bf413ebc44e8921791232f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.482990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.483632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.483911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.484785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.486511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.494754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2609996f15bf413ebc44e8921791232f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.495450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-30859058-55c1-4571-94c5-acb2be96647a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2609996f15bf413ebc44e8921791232f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:12.502163 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 812/1590] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:12 2022] DELETE /identity/v3/users/2609996f15bf413ebc44e8921791232f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:12.503640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35e364-348d-4300-8740-612d12a07fa6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.503924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35e364-348d-4300-8740-612d12a07fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.504365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35e364-348d-4300-8740-612d12a07fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.526174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1a35e364-348d-4300-8740-612d12a07fa6 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:12.527427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1a35e364-348d-4300-8740-612d12a07fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.593190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.596627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.617032 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 779/1591] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:12.629712 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 813/1592] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:12.634014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.635767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.655206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f16cd-d404-4f61-8b0f-757d6a38771a service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.663710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.664367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.664652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.664924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.665717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:12.666023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.668161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.681616 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 780/1593] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:12.685643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35e364-348d-4300-8740-612d12a07fa6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.695889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35e364-348d-4300-8740-612d12a07fa6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.701862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.702487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.702769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.703112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.704394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.705989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.759094 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 814/1594] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:12 2022] POST /identity/v3/projects => generated 401 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:12.763285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.778800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd5cd59-4269-41e1-9e01-ac7d831cfca4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.863985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:12.876654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.877413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.877795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.878421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.879152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.881195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.908359 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 781/1595] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:12 2022] POST /identity/v3/users => generated 301 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:12.911498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.927429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcc50f-4b98-45ea-995f-68092c3d9eb6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.928261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.930343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:12.941604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.944776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.945244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.945610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.946013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3be946-cccc-4dbc-be27-be6c2dbe6210 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:12.947638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.951502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:12.952258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:12.952399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.952562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:12.953405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] PATH_INFO: `/v3/projects/32979c839a884bf9883e21bf14fb4266` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:12.954971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32979c839a884bf9883e21bf14fb4266)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:12.957226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:12.980150 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 815/1596] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:12.983769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:12.994154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:13.001165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809b6eb-1c77-4aed-b21e-16a5507b3818 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.007888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.008539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.008826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.014277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.014277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:13.018015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:13.032035 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 816/1597] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:13.037017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.044450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32979c839a884bf9883e21bf14fb4266', 'request_id': 'req-2c641fe1-deb9-4686-a70b-d5360750cbf9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:13.051636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf166c76-97c6-43f0-8d32-d5b8f7258365 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.055245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32979c839a884bf9883e21bf14fb4266', 'request_id': 'req-2c641fe1-deb9-4686-a70b-d5360750cbf9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:13.064618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32979c839a884bf9883e21bf14fb4266', 'request_id': 'req-2c641fe1-deb9-4686-a70b-d5360750cbf9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:13.082968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] The token cache is being invalidate because project 32979c839a884bf9883e21bf14fb4266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:13.083479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:13.084483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2c641fe1-deb9-4686-a70b-d5360750cbf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:13.089674 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 782/1598] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:12 2022] DELETE /identity/v3/projects/32979c839a884bf9883e21bf14fb4266 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:13.095477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449d55-ccc2-4ac6-adae-3badd6724fea None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.096026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449d55-ccc2-4ac6-adae-3badd6724fea None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.096548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35449d55-ccc2-4ac6-adae-3badd6724fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.127113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-35449d55-ccc2-4ac6-adae-3badd6724fea None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:13.128288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-35449d55-ccc2-4ac6-adae-3badd6724fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:13.181634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44caf1-0987-461f-be47-8dfc797104b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.182653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44caf1-0987-461f-be47-8dfc797104b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.183191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44caf1-0987-461f-be47-8dfc797104b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.183606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44caf1-0987-461f-be47-8dfc797104b7 admin admin] PATH_INFO: `/v3/projects/780d3c9b7ec54efbb162414bbad4d49b/users/8824240f40ae4c6f8f2108ce8469f077/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.205303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44caf1-0987-461f-be47-8dfc797104b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780d3c9b7ec54efbb162414bbad4d49b, user_id=8824240f40ae4c6f8f2108ce8469f077, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:13.209070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44caf1-0987-461f-be47-8dfc797104b7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:13.234838 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 817/1599] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:13 2022] PUT /identity/v3/projects/780d3c9b7ec54efbb162414bbad4d49b/users/8824240f40ae4c6f8f2108ce8469f077/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:13.245510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.246560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.247731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.262103 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 783/1600] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:13.266990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e069da-8fd8-4852-8bf2-3d5f022047ce None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.267280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e069da-8fd8-4852-8bf2-3d5f022047ce None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.267558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e069da-8fd8-4852-8bf2-3d5f022047ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.291737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-82e069da-8fd8-4852-8bf2-3d5f022047ce None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:13.293234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-82e069da-8fd8-4852-8bf2-3d5f022047ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:13.323720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] MFA Rules not processed for user `8824240f40ae4c6f8f2108ce8469f077`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:13.327806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:13.460600 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 784/1601] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:13.465660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e069da-8fd8-4852-8bf2-3d5f022047ce None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.488180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e069da-8fd8-4852-8bf2-3d5f022047ce None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.508017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.508952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.509475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.509937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.510929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:13.511454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:13.514935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:13.519183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:13.556477 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 785/1602] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:13.561355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.581019 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 818/1603] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:13 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:13.586531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9bb8-57b5-4798-b771-33987e86b953 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.591950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.594707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:13.596964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.599689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.600113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.600629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.601313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:13.604850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:13.613737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464391d-d138-401d-98d7-ed43ef9bdbaf None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.618781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.619416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.619762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.620045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.620746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:13.622741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:13.645548 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 786/1604] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:13 2022] POST /identity/v3/projects => generated 391 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:13.647045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.650089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:13.686138 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 819/1605] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:13 2022] POST /identity/v3/projects => generated 399 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:13.687901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.764177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:13.790241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915a598-8a95-4e0d-9958-005f42400122 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.798585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.799781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.800244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.800862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.801721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:13.803982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:13.823223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353987a9-8ac3-4664-be6d-0a973d2104b0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.878961 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 820/1606] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:13 2022] POST /identity/v3/users => generated 296 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:13.880209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.902391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d51c2a-e961-4255-bef9-ee4f4d918553 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:13.913113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.913920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.914295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.916493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.917321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:13.919708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:13.975046 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 821/1607] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:13 2022] POST /identity/v3/users => generated 306 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:13.976018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:13.977314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:13.979153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:13.979438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:13.979692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:13.987291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:14.002054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840ca32c-a941-407b-a634-e08bf8f2873c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.013972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.017722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.018108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.018403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.019080 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.022296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.050796 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 822/1608] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:14.054722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.074569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.076274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.082058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e4800-24ff-413d-80fb-615a1d0e2c72 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.091481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.092029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.092328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.092603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.093253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.094185 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 787/1609] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:14.095373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.103321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.112038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a94da8-7e65-4d7b-a389-92b62be9a3bb service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.118592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.119316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.119526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.120264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.121966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.123613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.136105 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 823/1610] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:14.141167 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 788/1611] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:14.142018 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.147530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.168687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e1ae5-78c5-4702-90dd-8f14cb8e1d21 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.173535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf98331-ed84-4d93-8582-469c8900b5c9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.177010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.177709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.177978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.178260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] PATH_INFO: `/v3/projects/735759e2c5bc4379a06124a6224e6012/users/52fa89bf91d045498dcf65324802d9d5/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.186199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e57000d-d3a7-420e-be18-0ff41a24aea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.187327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57000d-d3a7-420e-be18-0ff41a24aea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.187787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57000d-d3a7-420e-be18-0ff41a24aea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.188334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e57000d-d3a7-420e-be18-0ff41a24aea2 admin admin] PATH_INFO: `/v3/projects/0276014e51074e2184113b778b81cf94/users/792c895b4d5443dabbd4a7912a26ab8d/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.212910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=735759e2c5bc4379a06124a6224e6012, user_id=52fa89bf91d045498dcf65324802d9d5, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.216072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.216959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57000d-d3a7-420e-be18-0ff41a24aea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0276014e51074e2184113b778b81cf94, user_id=792c895b4d5443dabbd4a7912a26ab8d, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.221560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e57000d-d3a7-420e-be18-0ff41a24aea2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.246872 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 824/1612] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:14 2022] PUT /identity/v3/projects/0276014e51074e2184113b778b81cf94/users/792c895b4d5443dabbd4a7912a26ab8d/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:14.257932 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 789/1613] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:14 2022] PUT /identity/v3/projects/735759e2c5bc4379a06124a6224e6012/users/52fa89bf91d045498dcf65324802d9d5/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:14.259587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700f481-281d-4132-9849-ce8d4d9603ec None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.260830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700f481-281d-4132-9849-ce8d4d9603ec None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.261114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700f481-281d-4132-9849-ce8d4d9603ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.262377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.287277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:14.316214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5700f481-281d-4132-9849-ce8d4d9603ec None None] MFA Rules not processed for user `792c895b4d5443dabbd4a7912a26ab8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:14.318482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5700f481-281d-4132-9849-ce8d4d9603ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:14.441889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e6b14-7c12-4563-9820-e5f915035157 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.459505 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 825/1614] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:14.460477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700f481-281d-4132-9849-ce8d4d9603ec None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.542594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.543567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.543567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.543861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.544538 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.546333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.562246 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 790/1615] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:14.563776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.579596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4b0d7a-bd21-4bba-9b7f-89b907bb28ac admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.587420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62b31e-6fd3-4dbe-84a5-cab77c7c5718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.587420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62b31e-6fd3-4dbe-84a5-cab77c7c5718 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.587420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62b31e-6fd3-4dbe-84a5-cab77c7c5718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.587420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62b31e-6fd3-4dbe-84a5-cab77c7c5718 admin admin] PATH_INFO: `/v3/projects/735759e2c5bc4379a06124a6224e6012/users/52fa89bf91d045498dcf65324802d9d5/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.589087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62b31e-6fd3-4dbe-84a5-cab77c7c5718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=735759e2c5bc4379a06124a6224e6012, user_id=52fa89bf91d045498dcf65324802d9d5, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.592700 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700f481-281d-4132-9849-ce8d4d9603ec None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.595103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62b31e-6fd3-4dbe-84a5-cab77c7c5718 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.601352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.602355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.603025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.603510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.606113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.609340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.618132 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 791/1616] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:14 2022] PUT /identity/v3/projects/735759e2c5bc4379a06124a6224e6012/users/52fa89bf91d045498dcf65324802d9d5/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:14.621262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70445d5-2a12-4124-8440-49ba8d256e89 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.621666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70445d5-2a12-4124-8440-49ba8d256e89 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.622012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70445d5-2a12-4124-8440-49ba8d256e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.624400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:14.662166 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 826/1617] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:14 2022] POST /identity/v3/projects => generated 393 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:14.666139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.684270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d70445d5-2a12-4124-8440-49ba8d256e89 None None] MFA Rules not processed for user `52fa89bf91d045498dcf65324802d9d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:14.685553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d70445d5-2a12-4124-8440-49ba8d256e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:14.759420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67b45a-de6b-4d82-9ca6-64bcd9b85ec3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.766715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.767350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.767632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.767907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.768581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.770509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.806174 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 827/1618] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:14 2022] POST /identity/v3/users => generated 303 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:14.807419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.819070 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 792/1619] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:14.825375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f062b-4960-4270-bdc9-c762a68870d6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.831121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70445d5-2a12-4124-8440-49ba8d256e89 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.844459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.845892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.845892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.846029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.846646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.848358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.861095 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 828/1620] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:14.865673 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.867158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:14.878774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0deb7-3dd3-4344-bc78-c2bd180ffe7b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.884558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.885410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.886464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.886464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] PATH_INFO: `/v3/projects/84a1a78ec272477782c5be5f186bdfe1/users/84015e571e0846f28b50d2676e7e7712/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.909893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a1a78ec272477782c5be5f186bdfe1, user_id=84015e571e0846f28b50d2676e7e7712, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.914791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.915342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70445d5-2a12-4124-8440-49ba8d256e89 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:14.919346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:14.920178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:14.920822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:14.920822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:14.942992 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 829/1621] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:14 2022] PUT /identity/v3/projects/84a1a78ec272477782c5be5f186bdfe1/users/84015e571e0846f28b50d2676e7e7712/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:14.944483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:14.954162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:14.955713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:14.989288 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 793/1622] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:15.061217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f96285-94f0-436f-ad65-f6ef34f254ce admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:15.078456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:15.079261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:15.079563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:15.079827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:15.080593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:15.082828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:15.096835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:15.109356 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 830/1623] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:15.110896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:15.114823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e00958-b748-4171-8444-1fa053a4414b service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:15.131484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2798fe60-3a06-47b6-aa35-83936678ac38 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:15.143745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7991a305-8c38-49a2-b8a6-baf1d5daa9de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:15.144476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991a305-8c38-49a2-b8a6-baf1d5daa9de admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:15.144760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991a305-8c38-49a2-b8a6-baf1d5daa9de admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:15.145034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7991a305-8c38-49a2-b8a6-baf1d5daa9de admin admin] PATH_INFO: `/v3/projects/84a1a78ec272477782c5be5f186bdfe1/users/84015e571e0846f28b50d2676e7e7712/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:15.150541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991a305-8c38-49a2-b8a6-baf1d5daa9de admin admin] RBAC: Authorizing `identity:create_grant(project_id=84a1a78ec272477782c5be5f186bdfe1, user_id=84015e571e0846f28b50d2676e7e7712, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:15.157198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7991a305-8c38-49a2-b8a6-baf1d5daa9de admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:15.204408 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 831/1624] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:15 2022] PUT /identity/v3/projects/84a1a78ec272477782c5be5f186bdfe1/users/84015e571e0846f28b50d2676e7e7712/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:15.207893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f595-becd-406e-9b67-827551876206 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:15.208026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f595-becd-406e-9b67-827551876206 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:15.208203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f624f595-becd-406e-9b67-827551876206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:15.251805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f624f595-becd-406e-9b67-827551876206 None None] MFA Rules not processed for user `84015e571e0846f28b50d2676e7e7712`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:15.253475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f624f595-becd-406e-9b67-827551876206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:15.260232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:15.261434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:15.262184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:15.262618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:15.268737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:15.430503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:15.433552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:15.450692 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 794/1625] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:15 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:15.489901 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 832/1626] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:15 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:15.738856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:15.741446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:15.771226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd431f1a-ac27-4d75-8f18-30df1ac1b688 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:15.901009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:15.904641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:15.905052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:15.906228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:15.931004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:15.934465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:15.960352 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 795/1627] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:15 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:16.071045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f595-becd-406e-9b67-827551876206 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:16.084130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f624f595-becd-406e-9b67-827551876206 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:16.089943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8106d672-afbf-4cff-8c3f-f1c6d0a3bb8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:16.090620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d672-afbf-4cff-8c3f-f1c6d0a3bb8d service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:16.091032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d672-afbf-4cff-8c3f-f1c6d0a3bb8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:16.091330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8106d672-afbf-4cff-8c3f-f1c6d0a3bb8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:16.219585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106d672-afbf-4cff-8c3f-f1c6d0a3bb8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:16.221122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8106d672-afbf-4cff-8c3f-f1c6d0a3bb8d service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:16.235409 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 833/1628] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:16 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:18.495579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:18.501133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:18.522528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:18.572537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63aa9d0-c131-4686-b1c6-5a714c6eb3e0 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:18.577561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:18.578768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:18.579043 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:18.579310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] PATH_INFO: `/v3/users/11ec83e5b20046e6aa48911663ce4a6b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:18.580825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11ec83e5b20046e6aa48911663ce4a6b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:18.582491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:18.636346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11ec83e5b20046e6aa48911663ce4a6b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:18.637209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11ec83e5b20046e6aa48911663ce4a6b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:18.645536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11ec83e5b20046e6aa48911663ce4a6b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:18.658931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11ec83e5b20046e6aa48911663ce4a6b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:18.659752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11ec83e5b20046e6aa48911663ce4a6b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:18.674565 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 796/1629] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:18 2022] DELETE /identity/v3/users/11ec83e5b20046e6aa48911663ce4a6b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:18.679280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:18.679734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:18.679854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:18.701483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:18.702684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:18.764659 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 834/1630] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:18.774641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:18.776080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:18.835495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b95eb7-a1c3-4e8c-9a0a-eb9e2f8e8844 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:18.839359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:18.840031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:18.840328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:18.840609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:18.850726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:18.852241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:18.863956 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 797/1631] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:19.647297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:19.648909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:19.663049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221244fe-0247-47d7-ae6f-a86b47db58f0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:19.678755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:19.679692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:19.681505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:19.682420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] PATH_INFO: `/v3/projects/d38a8c152f8d4d85b50912af027ff961` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:19.683556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d38a8c152f8d4d85b50912af027ff961)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:19.687804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:19.707923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:19.709333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:19.743131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bbab30-5d49-48ba-956c-fa710b045517 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:19.755218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:19.755960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:19.758594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:19.758594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] PATH_INFO: `/v3/users/792c895b4d5443dabbd4a7912a26ab8d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:19.761096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=792c895b4d5443dabbd4a7912a26ab8d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:19.776334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:19.828425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:19.845115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd38a8c152f8d4d85b50912af027ff961', 'request_id': 'req-77671593-5c16-4a8c-ba85-f40abf81ac57'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.852766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:19.863211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd38a8c152f8d4d85b50912af027ff961', 'request_id': 'req-77671593-5c16-4a8c-ba85-f40abf81ac57'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.864242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792c895b4d5443dabbd4a7912a26ab8d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.865344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792c895b4d5443dabbd4a7912a26ab8d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.876272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd38a8c152f8d4d85b50912af027ff961', 'request_id': 'req-77671593-5c16-4a8c-ba85-f40abf81ac57'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.878155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792c895b4d5443dabbd4a7912a26ab8d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.891897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] The token cache is being invalidate because project d38a8c152f8d4d85b50912af027ff961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:19.892184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.892757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792c895b4d5443dabbd4a7912a26ab8d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.893008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.893676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b5f3adce-e8f7-42bc-acae-d1914af56ed9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792c895b4d5443dabbd4a7912a26ab8d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:19.895119 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 835/1632] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:19 2022] DELETE /identity/v3/projects/d38a8c152f8d4d85b50912af027ff961 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:19.899351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:19.900757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:19.918693 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 798/1633] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:19 2022] DELETE /identity/v3/users/792c895b4d5443dabbd4a7912a26ab8d => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:19.923466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:19.923927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:19.924429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:19.947597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:19.948975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:19.985111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77671593-5c16-4a8c-ba85-f40abf81ac57 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:20.042413 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 799/1634] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:20.043361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:20.044833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:20.064269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cb3106-8b92-46c5-a5e3-8b22af32001b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:20.075121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:20.076605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:20.076880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:20.077185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:20.078286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] PATH_INFO: `/v3/users/204ad95a53e14c64bd4063e41e6daff4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:20.079180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:20.079458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:20.080245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] RBAC: Authorizing `identity:delete_user(user_id=204ad95a53e14c64bd4063e41e6daff4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:20.081150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] PATH_INFO: `/v3/users/8824240f40ae4c6f8f2108ce8469f077` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:20.082519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:20.084953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8824240f40ae4c6f8f2108ce8469f077)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:20.087099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:20.146237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '204ad95a53e14c64bd4063e41e6daff4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.155377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '204ad95a53e14c64bd4063e41e6daff4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.167143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '204ad95a53e14c64bd4063e41e6daff4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.183384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:20.186332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '204ad95a53e14c64bd4063e41e6daff4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.187799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '204ad95a53e14c64bd4063e41e6daff4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.197470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8824240f40ae4c6f8f2108ce8469f077'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.198193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8824240f40ae4c6f8f2108ce8469f077'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.201923 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 836/1635] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:19 2022] DELETE /identity/v3/users/204ad95a53e14c64bd4063e41e6daff4 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:20.203554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:20.205737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:20.225724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8824240f40ae4c6f8f2108ce8469f077'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.257178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8824240f40ae4c6f8f2108ce8469f077'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.257962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9fff451d-290a-45f1-bb00-6d93a4e8e31b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8824240f40ae4c6f8f2108ce8469f077'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.289618 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 800/1636] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:20 2022] DELETE /identity/v3/users/8824240f40ae4c6f8f2108ce8469f077 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:20.293489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:20.293780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:20.294324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:20.294604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0072b7-0616-48cc-8dc6-9ed76206d60d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:20.328840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:20.330152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:20.355973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:20.381818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:20.382202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:20.382634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:20.382917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:20.385454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:20.432783 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 801/1637] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:20.459054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:20.465179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:20.488608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:20.490148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:20.507918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5bc26b-dd28-4bc4-8e1f-06bba0d09bc8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:20.512896 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 837/1638] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:20.520684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:20.523032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:20.523332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:20.525763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:20.539576 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:20.541160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:20.556823 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 802/1639] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:20.789270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:20.791236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:20.819088 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb485fe-ce06-4a2d-a8e2-7482a023cefc service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:20.826880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:20.827838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:20.828343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:20.828806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] PATH_INFO: `/v3/projects/0276014e51074e2184113b778b81cf94` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:20.830258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0276014e51074e2184113b778b81cf94)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:20.832514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:20.916359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0276014e51074e2184113b778b81cf94', 'request_id': 'req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.933799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0276014e51074e2184113b778b81cf94', 'request_id': 'req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.936047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:20.948561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0276014e51074e2184113b778b81cf94', 'request_id': 'req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.974401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] The token cache is being invalidate because project 0276014e51074e2184113b778b81cf94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:20.974901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.976066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:20.978874 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 838/1640] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:20 2022] DELETE /identity/v3/projects/0276014e51074e2184113b778b81cf94 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:20.982884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:20.993863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0778bf7a-64af-45e7-bc35-058bfe2d2dfc service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:21.046006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:21.048234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:21.100819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:21.101638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:21.101922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:21.102193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] PATH_INFO: `/v3/users/84015e571e0846f28b50d2676e7e7712` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:21.103530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84015e571e0846f28b50d2676e7e7712)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:21.106938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:21.220932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c8f3c7-6174-4019-b891-d1d1f90bfe7e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:21.227075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:21.227739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:21.228018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:21.228295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] PATH_INFO: `/v3/projects/470454f897bb491aad98ed9a8f80e77d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:21.229484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=470454f897bb491aad98ed9a8f80e77d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:21.231419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:21.233055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84015e571e0846f28b50d2676e7e7712'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.234082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84015e571e0846f28b50d2676e7e7712'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.244278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84015e571e0846f28b50d2676e7e7712'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.262490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84015e571e0846f28b50d2676e7e7712'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.263800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84015e571e0846f28b50d2676e7e7712'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.285778 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 803/1641] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:20 2022] DELETE /identity/v3/users/84015e571e0846f28b50d2676e7e7712 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:21.302628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '470454f897bb491aad98ed9a8f80e77d', 'request_id': 'req-dba6b227-b76f-4a2d-856d-031b382fba15'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.316429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '470454f897bb491aad98ed9a8f80e77d', 'request_id': 'req-dba6b227-b76f-4a2d-856d-031b382fba15'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.328845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '470454f897bb491aad98ed9a8f80e77d', 'request_id': 'req-dba6b227-b76f-4a2d-856d-031b382fba15'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.350313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] The token cache is being invalidate because project 470454f897bb491aad98ed9a8f80e77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:21.350736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.351799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.354027 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 839/1642] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:21 2022] DELETE /identity/v3/projects/470454f897bb491aad98ed9a8f80e77d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:21.358370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:21.359891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:21.421057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:21.474287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fde22b-b69f-4653-b199-cf02204c9603 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:21.482532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:21.483197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:21.483476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:21.483809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] PATH_INFO: `/v3/users/ffacc79000b5460c91e1b9957d144024` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:21.485550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffacc79000b5460c91e1b9957d144024)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:21.488672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:21.583122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffacc79000b5460c91e1b9957d144024'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.585044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffacc79000b5460c91e1b9957d144024'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.600385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffacc79000b5460c91e1b9957d144024'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.639635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffacc79000b5460c91e1b9957d144024'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.643472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffacc79000b5460c91e1b9957d144024'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:21.676464 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 804/1643] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:21 2022] DELETE /identity/v3/users/ffacc79000b5460c91e1b9957d144024 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:21.800494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:21.805318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:21.899009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba6b227-b76f-4a2d-856d-031b382fba15 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:21.923702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:21.988493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:21.992900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:21.994368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:21.994368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] PATH_INFO: `/v3/projects/780d3c9b7ec54efbb162414bbad4d49b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:21.995835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780d3c9b7ec54efbb162414bbad4d49b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:21.999255 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.009374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.011487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.031069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cf39cb-0842-48a5-bd35-5eda5e768974 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.037519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.038519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.038676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.038985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] PATH_INFO: `/v3/projects/84a1a78ec272477782c5be5f186bdfe1` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.040737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] RBAC: Authorizing `identity:delete_project(project_id=84a1a78ec272477782c5be5f186bdfe1)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.053017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.060543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780d3c9b7ec54efbb162414bbad4d49b', 'request_id': 'req-10c37c77-939b-4a11-9d48-c414b02d47d7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.078856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780d3c9b7ec54efbb162414bbad4d49b', 'request_id': 'req-10c37c77-939b-4a11-9d48-c414b02d47d7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.089448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780d3c9b7ec54efbb162414bbad4d49b', 'request_id': 'req-10c37c77-939b-4a11-9d48-c414b02d47d7'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.102559 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] The token cache is being invalidate because project 780d3c9b7ec54efbb162414bbad4d49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:22.103660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.104518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.108387 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 840/1644] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:21 2022] DELETE /identity/v3/projects/780d3c9b7ec54efbb162414bbad4d49b => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:22.111456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.115229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.125975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84a1a78ec272477782c5be5f186bdfe1', 'request_id': 'req-bbf70dda-0948-486a-8507-4cc51c7a83be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.136680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84a1a78ec272477782c5be5f186bdfe1', 'request_id': 'req-bbf70dda-0948-486a-8507-4cc51c7a83be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.148842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84a1a78ec272477782c5be5f186bdfe1', 'request_id': 'req-bbf70dda-0948-486a-8507-4cc51c7a83be'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.161895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] The token cache is being invalidate because project 84a1a78ec272477782c5be5f186bdfe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:22.162171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.162913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.168992 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 805/1645] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:22 2022] DELETE /identity/v3/projects/84a1a78ec272477782c5be5f186bdfe1 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:22.168992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.171780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.229026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c37c77-939b-4a11-9d48-c414b02d47d7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.235347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:22.313019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf70dda-0948-486a-8507-4cc51c7a83be admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.318381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.319090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.319369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.319625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] PATH_INFO: `/v3/projects/4db827b5ab61413584a71d6bf9d5ddc7` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.321104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db827b5ab61413584a71d6bf9d5ddc7)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.322727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.335932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.336779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.337121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.337532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] PATH_INFO: `/v3/users/52fa89bf91d045498dcf65324802d9d5` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.339530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52fa89bf91d045498dcf65324802d9d5)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.341641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.344328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:22.379531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db827b5ab61413584a71d6bf9d5ddc7', 'request_id': 'req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.386468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db827b5ab61413584a71d6bf9d5ddc7', 'request_id': 'req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.393026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db827b5ab61413584a71d6bf9d5ddc7', 'request_id': 'req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.405324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52fa89bf91d045498dcf65324802d9d5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.405523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] The token cache is being invalidate because project 4db827b5ab61413584a71d6bf9d5ddc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:22.406387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.407056 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ab84c69-8cfa-4f8e-a481-4518caf6fa89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.408861 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 806/1646] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:22 2022] DELETE /identity/v3/projects/4db827b5ab61413584a71d6bf9d5ddc7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:22.413180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67bcc-bd95-43ff-a0cf-5bd5e1a5721f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.413343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67bcc-bd95-43ff-a0cf-5bd5e1a5721f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.413617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c67bcc-bd95-43ff-a0cf-5bd5e1a5721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.414108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52fa89bf91d045498dcf65324802d9d5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.423347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52fa89bf91d045498dcf65324802d9d5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.434897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-17c67bcc-bd95-43ff-a0cf-5bd5e1a5721f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:22.436113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-17c67bcc-bd95-43ff-a0cf-5bd5e1a5721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.436597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52fa89bf91d045498dcf65324802d9d5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.437402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fde0b9da-76bc-45f0-b335-940f3f80bff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52fa89bf91d045498dcf65324802d9d5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.444776 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 841/1647] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:22 2022] DELETE /identity/v3/users/52fa89bf91d045498dcf65324802d9d5 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:22.446121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f555454-6877-4d0a-a478-5539ef14855b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.446463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f555454-6877-4d0a-a478-5539ef14855b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.447052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f555454-6877-4d0a-a478-5539ef14855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.486516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2f555454-6877-4d0a-a478-5539ef14855b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:22.487788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2f555454-6877-4d0a-a478-5539ef14855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.529287 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 807/1648] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:22.537577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64eb04-b9ad-4b12-82a3-6ad30d1e797d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.537885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64eb04-b9ad-4b12-82a3-6ad30d1e797d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.538126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e64eb04-b9ad-4b12-82a3-6ad30d1e797d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.560983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1e64eb04-b9ad-4b12-82a3-6ad30d1e797d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:22.561663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1e64eb04-b9ad-4b12-82a3-6ad30d1e797d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.583157 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 842/1649] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:22.592189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4e315-f814-4b50-a8d0-dd0c8831218b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.593550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4e315-f814-4b50-a8d0-dd0c8831218b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.594358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4e315-f814-4b50-a8d0-dd0c8831218b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.626272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b6f4e315-f814-4b50-a8d0-dd0c8831218b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:22.627661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b6f4e315-f814-4b50-a8d0-dd0c8831218b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.638702 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 843/1650] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:22.643101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4e315-f814-4b50-a8d0-dd0c8831218b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.653564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4e315-f814-4b50-a8d0-dd0c8831218b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.654436 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 808/1651] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:22.659241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.659809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64eb04-b9ad-4b12-82a3-6ad30d1e797d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.659918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.660144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.660454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.661185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:22.661632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.663383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.668971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:22.673777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e64eb04-b9ad-4b12-82a3-6ad30d1e797d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.677875 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 844/1652] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:56:22.683417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.683600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.685061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.685583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.686216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.687754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:22.689015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.692482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.702919 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fc1b60-e532-4dc3-b687-af37c63fe394 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.716661 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 809/1653] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:22.718094 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.719380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.720364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.720656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.720929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.721766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.727841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.746787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6902404-f036-49cc-ab6a-e80f78e898bd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.759344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.760459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.760897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.761311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.762102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.764077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.800348 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 845/1654] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:22.804062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.828294 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 810/1655] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:22.829311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.830968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:22.885692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:22.901518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290a6ba2-3aa1-4c55-ae35-fe3fc4383352 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.909801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.911074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.911351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.911618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] PATH_INFO: `/v3/projects/735759e2c5bc4379a06124a6224e6012` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.913047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=735759e2c5bc4379a06124a6224e6012)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.914478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6fc222-1396-420a-b007-abbef7d6e346 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.915883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.919062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:22.919668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:22.919946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:22.920214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:22.920882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:22.922813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:22.953481 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 846/1656] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:22.954981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:22.966467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9600b0d6-d66c-4c17-8fa8-cc6751f5a5a1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:22.977787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735759e2c5bc4379a06124a6224e6012', 'request_id': 'req-c06348fe-55a5-4a4a-b899-3f63a6134b3c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.987000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735759e2c5bc4379a06124a6224e6012', 'request_id': 'req-c06348fe-55a5-4a4a-b899-3f63a6134b3c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:22.993313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735759e2c5bc4379a06124a6224e6012', 'request_id': 'req-c06348fe-55a5-4a4a-b899-3f63a6134b3c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:23.005340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] The token cache is being invalidate because project 735759e2c5bc4379a06124a6224e6012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:23.005632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:23.006308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:23.008038 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 811/1657] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:22 2022] DELETE /identity/v3/projects/735759e2c5bc4379a06124a6224e6012 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:23.008754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.010062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:23.044351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd90f6a-58fd-42ac-9ed9-789674d4056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.044999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90f6a-58fd-42ac-9ed9-789674d4056b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.045274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90f6a-58fd-42ac-9ed9-789674d4056b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.045548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd90f6a-58fd-42ac-9ed9-789674d4056b admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.046223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd90f6a-58fd-42ac-9ed9-789674d4056b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.048504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd90f6a-58fd-42ac-9ed9-789674d4056b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.073285 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 847/1658] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:22 2022] POST /identity/v3/users => generated 291 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:23.078510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a95f4-dd44-4399-bdea-a6d2bf44bfe8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.078824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a95f4-dd44-4399-bdea-a6d2bf44bfe8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.079066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a95f4-dd44-4399-bdea-a6d2bf44bfe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.085492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06348fe-55a5-4a4a-b899-3f63a6134b3c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.090387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.091152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.091434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.091700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.092703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.094895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.096643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:23.101872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c41a95f4-dd44-4399-bdea-a6d2bf44bfe8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:23.103167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c41a95f4-dd44-4399-bdea-a6d2bf44bfe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:23.114223 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 812/1659] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:23.115464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.120477 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 848/1660] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:23.123018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3d13-b4db-483d-9de0-65c3c9eb3a12 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.123800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3d13-b4db-483d-9de0-65c3c9eb3a12 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.124084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f3d13-b4db-483d-9de0-65c3c9eb3a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.125750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:23.141667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e97902-7146-40cd-9321-c4b2146c02c6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.146539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.147275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.147620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.148006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.148796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.149583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-869f3d13-b4db-483d-9de0-65c3c9eb3a12 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:23.150541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.150863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-869f3d13-b4db-483d-9de0-65c3c9eb3a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:23.160058 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 849/1661] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:23.160875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f3d13-b4db-483d-9de0-65c3c9eb3a12 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.168771 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 813/1662] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:23.169609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.171839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f3d13-b4db-483d-9de0-65c3c9eb3a12 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.177256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.177442 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.177554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.177829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.178499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.180072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.181161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e751a375-aa07-4940-b5c9-b81301e95a48 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.185830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.186536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.186912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.187250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.187965 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:23.188361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.190003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.191892 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 850/1663] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:23.192666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.202782 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1d6fe3-616a-4550-85b3-273aeceac04f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.206590 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 814/1664] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:23.208171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.208904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.209254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.209604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.210572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.210741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.212177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.224727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5bf95-8c6a-4a5d-805f-a29d69a0d572 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.227897 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 851/1665] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:23.228841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.231482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.232477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.232754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.233020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] PATH_INFO: `/v3/projects/07941785917446599d8b991944f7de9d/users/08616a72155d4b8595bb7d813b155ad0/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.241273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d2863-7e31-4cb9-a7ec-cf6787bfe06b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.246567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d128b3-35ea-4393-8730-f16bddcfdfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.247292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d128b3-35ea-4393-8730-f16bddcfdfea admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.247569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d128b3-35ea-4393-8730-f16bddcfdfea admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.247960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d128b3-35ea-4393-8730-f16bddcfdfea admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.248715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d128b3-35ea-4393-8730-f16bddcfdfea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.250649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d128b3-35ea-4393-8730-f16bddcfdfea admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.252383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07941785917446599d8b991944f7de9d, user_id=08616a72155d4b8595bb7d813b155ad0, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.255790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.276923 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 815/1666] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:23 2022] PUT /identity/v3/projects/07941785917446599d8b991944f7de9d/users/08616a72155d4b8595bb7d813b155ad0/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:23.277763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.305554 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 852/1667] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:23.311488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e68526-1834-4c4b-84bb-f9057d0a3542 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.317199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e68526-1834-4c4b-84bb-f9057d0a3542 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.318749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e68526-1834-4c4b-84bb-f9057d0a3542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.367709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a4e68526-1834-4c4b-84bb-f9057d0a3542 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:23.367709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a4e68526-1834-4c4b-84bb-f9057d0a3542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:23.392079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2450ba-cf33-4da2-a889-0042c6ef7af7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.461228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:23.482136 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 853/1668] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:23.483344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e68526-1834-4c4b-84bb-f9057d0a3542 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.499164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e68526-1834-4c4b-84bb-f9057d0a3542 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.500869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.501854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.502364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.503078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] PATH_INFO: `/v3/projects/82e9271e843e46fb8875de2059d41f5d/users/a0cba5dfb0c54a49ba1654ad659e7f4a/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.505222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.506068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.506666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.506941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.508893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.510921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.526737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=82e9271e843e46fb8875de2059d41f5d, user_id=a0cba5dfb0c54a49ba1654ad659e7f4a, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.531512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.550815 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 854/1670] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/users => generated 286 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:23.552641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.552922 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 816/1670] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:23 2022] PUT /identity/v3/projects/82e9271e843e46fb8875de2059d41f5d/users/a0cba5dfb0c54a49ba1654ad659e7f4a/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:23.553916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.555224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:23.601210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:23.626943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572644a1-871a-4a07-908c-249bb1d434bd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.632200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c741ff-7d0a-4869-a5cd-576808ec6f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.632848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c741ff-7d0a-4869-a5cd-576808ec6f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.633124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c741ff-7d0a-4869-a5cd-576808ec6f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.633396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c741ff-7d0a-4869-a5cd-576808ec6f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.634039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c741ff-7d0a-4869-a5cd-576808ec6f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.636526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c741ff-7d0a-4869-a5cd-576808ec6f05 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.650481 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 817/1671] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:23.650481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaee0e-1a6d-4d2b-8e8a-7af953eb01b9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.650481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaee0e-1a6d-4d2b-8e8a-7af953eb01b9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.650804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdaee0e-1a6d-4d2b-8e8a-7af953eb01b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.651788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61afbe-8c07-430c-aeca-771d01667f34 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.688057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7bdaee0e-1a6d-4d2b-8e8a-7af953eb01b9 None None] MFA Rules not processed for user `a0cba5dfb0c54a49ba1654ad659e7f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:23.689809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7bdaee0e-1a6d-4d2b-8e8a-7af953eb01b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:23.714443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.715318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.715693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.716139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.728364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.730826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.744513 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 855/1672] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:23.745424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.770039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afad27-1578-4175-9903-d3b59f78619a service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.775926 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 818/1673] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:23.781476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaee0e-1a6d-4d2b-8e8a-7af953eb01b9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.783485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.784117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.784425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.784701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.785345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.788646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.794473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdaee0e-1a6d-4d2b-8e8a-7af953eb01b9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.801776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.802459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.802745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.803067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.803721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.806490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.827580 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 856/1674] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:23.832879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.850063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc0728-6e96-4c86-82bf-c48709f7188d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:23.864806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751a740-5042-46c5-bd45-b644655317b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:23.866067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751a740-5042-46c5-bd45-b644655317b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.866609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751a740-5042-46c5-bd45-b644655317b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.868388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751a740-5042-46c5-bd45-b644655317b3 admin admin] PATH_INFO: `/v3/projects/fbb04f1a5b184c9eb995165da5a3e58a/users/58f79424c9154a759bb602f692655bbc/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.885496 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 819/1675] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:23.889498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:23.905007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751a740-5042-46c5-bd45-b644655317b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbb04f1a5b184c9eb995165da5a3e58a, user_id=58f79424c9154a759bb602f692655bbc, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:23.909537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b751a740-5042-46c5-bd45-b644655317b3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:23.928949 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 857/1676] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:23 2022] PUT /identity/v3/projects/fbb04f1a5b184c9eb995165da5a3e58a/users/58f79424c9154a759bb602f692655bbc/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:23.933838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf1082-fee5-47b1-8711-ec5bf4c61eb8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:23.934402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf1082-fee5-47b1-8711-ec5bf4c61eb8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:23.935050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf1082-fee5-47b1-8711-ec5bf4c61eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:23.964503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a2cf1082-fee5-47b1-8711-ec5bf4c61eb8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:23.966397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a2cf1082-fee5-47b1-8711-ec5bf4c61eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:24.061137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4269266-b130-426e-82f9-f032f3f66c1a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:24.095685 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 858/1677] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:24.138469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf1082-fee5-47b1-8711-ec5bf4c61eb8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:24.156365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:24.176658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:24.197268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:24.198188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.198661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.199137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.199967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:24.203136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:24.235682 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 820/1678] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:23 2022] POST /identity/v3/users => generated 296 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:24.236048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf1082-fee5-47b1-8711-ec5bf4c61eb8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:24.240262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:24.240565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:24.241074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.242145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.242145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.267432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:24.268161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c067b0b-8226-4ed4-9bdf-c752658451f2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:24.268913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:24.280589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:24.281228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.282239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.282507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.283168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:24.284807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:24.296726 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 859/1679] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:24.304420 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 821/1680] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:24.308711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:24.333484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7722e837-a35a-405d-b6a2-aa9d97778aff service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:24.349883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:24.351601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.352548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.353014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] PATH_INFO: `/v3/projects/08b7f4983c68410aa6fc3060c30e8971/users/abd7802389ea4b3dbe6adee1ff771945/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.381730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b7f4983c68410aa6fc3060c30e8971, user_id=abd7802389ea4b3dbe6adee1ff771945, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:24.386761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:24.413014 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 860/1681] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:24 2022] PUT /identity/v3/projects/08b7f4983c68410aa6fc3060c30e8971/users/abd7802389ea4b3dbe6adee1ff771945/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:24.417328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:24.435418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e2e5a-fbf8-4438-ba98-75c6b7a17209 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:24.541930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea67638-936e-4398-bb97-b0e0ffed448e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:24.542573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea67638-936e-4398-bb97-b0e0ffed448e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.542950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea67638-936e-4398-bb97-b0e0ffed448e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.543286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea67638-936e-4398-bb97-b0e0ffed448e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.543991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea67638-936e-4398-bb97-b0e0ffed448e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:24.545834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea67638-936e-4398-bb97-b0e0ffed448e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:24.560288 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 822/1682] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:24 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:24.565113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:24.580445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021aab7-e3b4-49db-bb9b-5faff6cd8cad admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:24.587305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a818c-de91-407f-8335-88c17359303b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:24.588787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a818c-de91-407f-8335-88c17359303b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.589747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a818c-de91-407f-8335-88c17359303b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.590225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a818c-de91-407f-8335-88c17359303b admin admin] PATH_INFO: `/v3/projects/08b7f4983c68410aa6fc3060c30e8971/users/abd7802389ea4b3dbe6adee1ff771945/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.592845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a818c-de91-407f-8335-88c17359303b admin admin] RBAC: Authorizing `identity:create_grant(project_id=08b7f4983c68410aa6fc3060c30e8971, user_id=abd7802389ea4b3dbe6adee1ff771945, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:24.597191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a818c-de91-407f-8335-88c17359303b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:24.640539 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 861/1683] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:24 2022] PUT /identity/v3/projects/08b7f4983c68410aa6fc3060c30e8971/users/abd7802389ea4b3dbe6adee1ff771945/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:24.654306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddaac-edd3-4998-aa30-cbe79e4d6b9a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.654649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddaac-edd3-4998-aa30-cbe79e4d6b9a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.654862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fddaac-edd3-4998-aa30-cbe79e4d6b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.690527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-07fddaac-edd3-4998-aa30-cbe79e4d6b9a None None] MFA Rules not processed for user `abd7802389ea4b3dbe6adee1ff771945`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:24.691850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-07fddaac-edd3-4998-aa30-cbe79e4d6b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:24.781184 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 823/1684] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:24.791989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a818c-de91-407f-8335-88c17359303b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:24.794217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-789a818c-de91-407f-8335-88c17359303b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:24.894843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a818c-de91-407f-8335-88c17359303b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:24.897266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b19f69-fe6f-4732-b1f7-0ea393d2628f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.897527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b19f69-fe6f-4732-b1f7-0ea393d2628f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.897802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b19f69-fe6f-4732-b1f7-0ea393d2628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.920983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:24.922297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:24.924034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:24.924501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:24.951981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:24.957083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:24.964068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e4b19f69-fe6f-4732-b1f7-0ea393d2628f None None] MFA Rules not processed for user `08616a72155d4b8595bb7d813b155ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:24.965327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e4b19f69-fe6f-4732-b1f7-0ea393d2628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:24.985755 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 862/1685] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:25.094693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:25.165320 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 824/1686] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:25.176784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:25.186462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c2762-3f88-4b99-ade5-77b9edb7b5e3 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:25.197904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:25.200691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:25.201198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:25.202486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:25.226783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:25.231293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:25.257762 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 863/1687] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:25.385587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458768-eed8-4d0d-a3a3-18ab62eba80e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:25.386076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458768-eed8-4d0d-a3a3-18ab62eba80e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:25.386495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24458768-eed8-4d0d-a3a3-18ab62eba80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:25.436630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-24458768-eed8-4d0d-a3a3-18ab62eba80e None None] MFA Rules not processed for user `58f79424c9154a759bb602f692655bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:25.438371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-24458768-eed8-4d0d-a3a3-18ab62eba80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:25.594490 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 825/1688] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:25.598643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:25.720405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f6ccf-994f-4bdb-9660-25f051903606 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:25.743749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24458768-eed8-4d0d-a3a3-18ab62eba80e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:25.753966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:25.754306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:25.758388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24458768-eed8-4d0d-a3a3-18ab62eba80e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:25.758544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:25.758847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:25.759443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:25.764262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:25.765604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:25.766029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:25.766451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] PATH_INFO: `/v3/users/a0cba5dfb0c54a49ba1654ad659e7f4a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:25.767801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0cba5dfb0c54a49ba1654ad659e7f4a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:25.768770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:25.770077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:25.781365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:25.825101 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 864/1689] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:25 2022] POST /identity/v3/projects => generated 369 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:25.831988 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:25.868067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0cba5dfb0c54a49ba1654ad659e7f4a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:25.869123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0cba5dfb0c54a49ba1654ad659e7f4a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:25.878885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0cba5dfb0c54a49ba1654ad659e7f4a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:25.904259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0cba5dfb0c54a49ba1654ad659e7f4a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:25.905424 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-c29f0868-6e20-4315-9ea0-b96a7b358124 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0cba5dfb0c54a49ba1654ad659e7f4a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:25.935651 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 826/1690] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:25 2022] DELETE /identity/v3/users/a0cba5dfb0c54a49ba1654ad659e7f4a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:25.938546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:25.938880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:25.939160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:25.960984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:25.962349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:25.975485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf74a293-2e8f-4e7d-b4c1-4ab00d3cc258 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:26.004026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:26.045216 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 827/1691] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:26.053116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:26.054872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:26.055154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:26.055474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:26.056148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:26.057448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:26.059265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:26.059475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:26.086820 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 865/1692] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:25 2022] POST /identity/v3/users => generated 291 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:26.088949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:26.091519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:26.104209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81761501-dccc-4755-a047-9c8f188c110c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:26.112231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:26.112956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:26.113246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:26.113523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:26.114173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:26.115933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:26.128338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 866/1693] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:26.131321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:26.142711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89166ca7-bb07-4759-b233-d7de5d413a66 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:26.146676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244296bf-4244-4d28-83f1-86bc1223f419 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:26.148850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:26.149685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:26.149972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:26.150336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] PATH_INFO: `/v3/projects/f8692c5b86f64ab1a98ea2dd6c7cffbe/users/587f3da8b3534260bb3bc3072bb68983/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:26.150939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:26.151620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:26.151966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:26.152393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:26.163664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:26.165378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:26.174616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8692c5b86f64ab1a98ea2dd6c7cffbe, user_id=587f3da8b3534260bb3bc3072bb68983, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:26.177835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:26.183545 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 828/1694] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:26.198498 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 867/1695] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:26 2022] PUT /identity/v3/projects/f8692c5b86f64ab1a98ea2dd6c7cffbe/users/587f3da8b3534260bb3bc3072bb68983/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:26.210005 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:26.224152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23386c5-179c-4679-b374-46cc89d74de5 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:26.329322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a968e4-2f63-4e8f-a6fe-4a2f8b0cc120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:26.330014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a968e4-2f63-4e8f-a6fe-4a2f8b0cc120 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:26.330372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a968e4-2f63-4e8f-a6fe-4a2f8b0cc120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:26.331669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a968e4-2f63-4e8f-a6fe-4a2f8b0cc120 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:26.332631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a968e4-2f63-4e8f-a6fe-4a2f8b0cc120 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:26.335920 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a968e4-2f63-4e8f-a6fe-4a2f8b0cc120 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:26.352982 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 829/1696] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:26 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:26.357806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:26.373658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f1cf8-cdb5-410e-8a06-4917557ed7a7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:26.384331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:26.386629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:26.386629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:26.386830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] PATH_INFO: `/v3/projects/f8692c5b86f64ab1a98ea2dd6c7cffbe/users/587f3da8b3534260bb3bc3072bb68983/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:26.389133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8692c5b86f64ab1a98ea2dd6c7cffbe, user_id=587f3da8b3534260bb3bc3072bb68983, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:26.393368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:26.420623 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 868/1697] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:26 2022] PUT /identity/v3/projects/f8692c5b86f64ab1a98ea2dd6c7cffbe/users/587f3da8b3534260bb3bc3072bb68983/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:26.914331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f60a9-c24d-46d0-9f98-e2668a8718de None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:26.914994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f60a9-c24d-46d0-9f98-e2668a8718de None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:26.915545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756f60a9-c24d-46d0-9f98-e2668a8718de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:26.942145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-756f60a9-c24d-46d0-9f98-e2668a8718de None None] MFA Rules not processed for user `08616a72155d4b8595bb7d813b155ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:26.947801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-756f60a9-c24d-46d0-9f98-e2668a8718de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:27.019087 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 830/1698] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:27.256709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:27.258604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:27.326434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:27.400359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e08f08f-2ba6-484b-9f22-8499824ef4d3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:27.416760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:27.419004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:27.421278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:27.421941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] PATH_INFO: `/v3/projects/82e9271e843e46fb8875de2059d41f5d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:27.425843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82e9271e843e46fb8875de2059d41f5d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:27.432179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:27.531294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82e9271e843e46fb8875de2059d41f5d', 'request_id': 'req-f1b2e354-ac61-4929-a571-cb4ea638eb44'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.547030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82e9271e843e46fb8875de2059d41f5d', 'request_id': 'req-f1b2e354-ac61-4929-a571-cb4ea638eb44'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.565155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82e9271e843e46fb8875de2059d41f5d', 'request_id': 'req-f1b2e354-ac61-4929-a571-cb4ea638eb44'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.582190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] The token cache is being invalidate because project 82e9271e843e46fb8875de2059d41f5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:27.583160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.583650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b2e354-ac61-4929-a571-cb4ea638eb44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.588412 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 869/1699] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:27 2022] DELETE /identity/v3/projects/82e9271e843e46fb8875de2059d41f5d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:27.590603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f60a9-c24d-46d0-9f98-e2668a8718de None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:27.605019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756f60a9-c24d-46d0-9f98-e2668a8718de None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:27.634781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:27.715989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:27.717439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:27.717976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:27.719212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] PATH_INFO: `/v3/users/abd7802389ea4b3dbe6adee1ff771945` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:27.721826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd7802389ea4b3dbe6adee1ff771945)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:27.724098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:27.837456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd7802389ea4b3dbe6adee1ff771945'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.837780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd7802389ea4b3dbe6adee1ff771945'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.851507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd7802389ea4b3dbe6adee1ff771945'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.930192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd7802389ea4b3dbe6adee1ff771945'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.930192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-67c32368-a1f8-4e9a-b5aa-031288ca5ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd7802389ea4b3dbe6adee1ff771945'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:27.935313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:27.935602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:27.936027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:27.952770 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 831/1700] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:27 2022] DELETE /identity/v3/users/abd7802389ea4b3dbe6adee1ff771945 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:27.996781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] MFA Rules not processed for user `587f3da8b3534260bb3bc3072bb68983`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:27.998173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:28.118001 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 870/1701] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:28.129645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:28.129936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:28.130188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:28.161952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] MFA Rules not processed for user `58f79424c9154a759bb602f692655bbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:28.163718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:28.263565 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 832/1702] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:28.316795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:28.319253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:28.391150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:28.396734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:28.398356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:28.424650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856c35da-8c08-436f-a63e-4d39d63dc695 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:28.431095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3809bf5-66ef-4b3d-bdf8-b5de44faf24c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:28.431735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3809bf5-66ef-4b3d-bdf8-b5de44faf24c service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:28.431975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3809bf5-66ef-4b3d-bdf8-b5de44faf24c service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:28.432949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3809bf5-66ef-4b3d-bdf8-b5de44faf24c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:28.434243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d3809bf5-66ef-4b3d-bdf8-b5de44faf24c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:28.446048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3809bf5-66ef-4b3d-bdf8-b5de44faf24c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:28.447750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3809bf5-66ef-4b3d-bdf8-b5de44faf24c service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:28.462183 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 871/1703] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:28.478352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:28.518273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64051ed-e8af-40eb-9a22-66573e9e4e97 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:28.523607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:28.524436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:28.524922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:28.525266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] PATH_INFO: `/v3/projects/08b7f4983c68410aa6fc3060c30e8971` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:28.527082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08b7f4983c68410aa6fc3060c30e8971)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:28.529399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:28.631213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08b7f4983c68410aa6fc3060c30e8971', 'request_id': 'req-d53f25c3-946b-4747-8eb3-cd5a39854685'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:28.645333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08b7f4983c68410aa6fc3060c30e8971', 'request_id': 'req-d53f25c3-946b-4747-8eb3-cd5a39854685'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:28.656016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08b7f4983c68410aa6fc3060c30e8971', 'request_id': 'req-d53f25c3-946b-4747-8eb3-cd5a39854685'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:28.669643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] The token cache is being invalidate because project 08b7f4983c68410aa6fc3060c30e8971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:28.669931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:28.670605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d53f25c3-946b-4747-8eb3-cd5a39854685 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:28.675671 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 833/1704] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:28 2022] DELETE /identity/v3/projects/08b7f4983c68410aa6fc3060c30e8971 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:28.678503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6411c4f-edeb-4836-aded-e5f45d6dc446 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:28.692429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6411c4f-edeb-4836-aded-e5f45d6dc446 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:28.692429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6411c4f-edeb-4836-aded-e5f45d6dc446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:28.721478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d6411c4f-edeb-4836-aded-e5f45d6dc446 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:28.722748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d6411c4f-edeb-4836-aded-e5f45d6dc446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:28.808346 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 872/1705] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:28.813941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3069d72-6c1f-486f-8cce-0bb9bf365470 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:28.814590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3069d72-6c1f-486f-8cce-0bb9bf365470 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:28.814875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3069d72-6c1f-486f-8cce-0bb9bf365470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:28.848443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d3069d72-6c1f-486f-8cce-0bb9bf365470 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:28.849804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d3069d72-6c1f-486f-8cce-0bb9bf365470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:28.866489 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 834/1706] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:28.871603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6411c4f-edeb-4836-aded-e5f45d6dc446 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:28.883004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6411c4f-edeb-4836-aded-e5f45d6dc446 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:28.889349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:28.890605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:28.890897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:28.891518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:28.892145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:28.892528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:28.894260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:28.918108 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 873/1707] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:28.923051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3069d72-6c1f-486f-8cce-0bb9bf365470 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:28.940347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3069d72-6c1f-486f-8cce-0bb9bf365470 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:28.951203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:28.951839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:28.952115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:28.955674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:28.956368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:28.958287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:29.002017 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 835/1708] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:28 2022] POST /identity/v3/projects => generated 375 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:29.007160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:29.017186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814f90c-0882-40c9-9ca7-9812cd197e75 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:29.077976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:29.078733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:29.079024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:29.079465 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:29.080108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:29.082062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:29.105530 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 874/1709] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:29 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:29.109282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:29.120003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3ec4b-9e57-4683-9066-e052db78a8be admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:29.125265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:29.126217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:29.126306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:29.126517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:29.127158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:29.129022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:29.148424 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 836/1710] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:29.148611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:29.156433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba5113-31b8-41dd-baf7-66ff149fb521 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:29.161487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd2a0b-838f-471a-bdf4-e54806369a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:29.162119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd2a0b-838f-471a-bdf4-e54806369a63 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:29.162403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd2a0b-838f-471a-bdf4-e54806369a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:29.162679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd2a0b-838f-471a-bdf4-e54806369a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:29.163350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd2a0b-838f-471a-bdf4-e54806369a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:29.165054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd2a0b-838f-471a-bdf4-e54806369a63 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:29.177466 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 875/1711] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:29.181668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:29.193562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0767c-2365-49e5-a42f-93bf33304cf5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:29.198548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:29.199234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:29.199512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:29.199785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] PATH_INFO: `/v3/projects/0d752c4b92b6429382c11751ad11b422/users/c6e35aeae88d43a6b4ccf7375021ede5/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:29.232520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d752c4b92b6429382c11751ad11b422, user_id=c6e35aeae88d43a6b4ccf7375021ede5, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:29.235689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:29.258045 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 837/1712] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:29 2022] PUT /identity/v3/projects/0d752c4b92b6429382c11751ad11b422/users/c6e35aeae88d43a6b4ccf7375021ede5/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:29.270444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8153e5-5d7d-4760-adbf-93d3b43b4488 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:29.270980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8153e5-5d7d-4760-adbf-93d3b43b4488 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:29.271396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8153e5-5d7d-4760-adbf-93d3b43b4488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:29.331527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2e8153e5-5d7d-4760-adbf-93d3b43b4488 None None] MFA Rules not processed for user `c6e35aeae88d43a6b4ccf7375021ede5`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:29.332920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2e8153e5-5d7d-4760-adbf-93d3b43b4488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:29.374968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:29.432911 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 876/1713] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:29.441371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:29.443130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:29.520795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473802a6-918f-458b-b9c5-28eff9b2fc9e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:29.526208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bd7f03-3c42-4266-b665-70dd2ff5d763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:29.526927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd7f03-3c42-4266-b665-70dd2ff5d763 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:29.527304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd7f03-3c42-4266-b665-70dd2ff5d763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:29.527497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bd7f03-3c42-4266-b665-70dd2ff5d763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:29.537676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd7f03-3c42-4266-b665-70dd2ff5d763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:29.539177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bd7f03-3c42-4266-b665-70dd2ff5d763 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:29.552514 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 838/1714] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:39.427155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=13 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:39.547060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:43.553980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1f5f-c888-4c89-a6d5-ae9e5f6bf61a None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:43.554899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1f5f-c888-4c89-a6d5-ae9e5f6bf61a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:43.555366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd1f5f-c888-4c89-a6d5-ae9e5f6bf61a None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:43.557111 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 877/1715] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:56:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:56:43.569282 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:43.569707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:43.569707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:43.591683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:43.593008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:43.657084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd1f5f-c888-4c89-a6d5-ae9e5f6bf61a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:43.659012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a8dd1f5f-c888-4c89-a6d5-ae9e5f6bf61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:43.676500 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 839/1716] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:56:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:43.700087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:43.701588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:43.714299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b134470-fcb1-4ef6-9ad8-ae1ee281cc51 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:43.720393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:43.721003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:43.721284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:43.721560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:43.730057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:43.731702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:43.742103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd1f5f-c888-4c89-a6d5-ae9e5f6bf61a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:43.742784 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 840/1717] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:43.746822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:43.747591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:43.747878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:43.748153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] PATH_INFO: `/v3/users/902cd162a23540d7b458a9f8b432b6c4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:43.750165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=902cd162a23540d7b458a9f8b432b6c4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:43.752078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:43.804666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '902cd162a23540d7b458a9f8b432b6c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:43.814328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '902cd162a23540d7b458a9f8b432b6c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:43.828134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '902cd162a23540d7b458a9f8b432b6c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:43.843159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '902cd162a23540d7b458a9f8b432b6c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:43.843978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '902cd162a23540d7b458a9f8b432b6c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:43.854359 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 878/1718] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:43 2022] DELETE /identity/v3/users/902cd162a23540d7b458a9f8b432b6c4 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:44.022963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.035258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee8545d-e185-4ee4-96d6-3dc18a643ea4 service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.102606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.103256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.103536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.103801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] PATH_INFO: `/v3/users/c6e35aeae88d43a6b4ccf7375021ede5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.104986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e35aeae88d43a6b4ccf7375021ede5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.106908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.170697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e35aeae88d43a6b4ccf7375021ede5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.172057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e35aeae88d43a6b4ccf7375021ede5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.181319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e35aeae88d43a6b4ccf7375021ede5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.205383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.206808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:44.222706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e35aeae88d43a6b4ccf7375021ede5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.222923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b699d01-348c-4f7c-950f-523460891bb3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.223715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e1414a9-dad6-4e8d-8ff0-c6c0ccd1a1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e35aeae88d43a6b4ccf7375021ede5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.229260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.230037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.230334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.230660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] PATH_INFO: `/v3/projects/c7e74b2109214d21bb658dc423b38505` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.231845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7e74b2109214d21bb658dc423b38505)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.235136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.240146 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 841/1719] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:44 2022] DELETE /identity/v3/users/c6e35aeae88d43a6b4ccf7375021ede5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:44.244402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.244909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.245311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.254859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:44.270809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:44.272924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:44.296406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7e74b2109214d21bb658dc423b38505', 'request_id': 'req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.303876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7e74b2109214d21bb658dc423b38505', 'request_id': 'req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.310512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7e74b2109214d21bb658dc423b38505', 'request_id': 'req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.322209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] The token cache is being invalidate because project c7e74b2109214d21bb658dc423b38505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:44.322505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.323151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e140a0f3-3309-4aa0-aa3e-2d856b843e4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:44.324662 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 879/1720] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:44 2022] DELETE /identity/v3/projects/c7e74b2109214d21bb658dc423b38505 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:44.330772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84313cf-6534-496e-b3f1-a7d350c97e0b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.331177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84313cf-6534-496e-b3f1-a7d350c97e0b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.331453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84313cf-6534-496e-b3f1-a7d350c97e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.334412 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 842/1721] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:44.339767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:44.344532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.345871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:44.355657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e84313cf-6534-496e-b3f1-a7d350c97e0b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:44.356895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e84313cf-6534-496e-b3f1-a7d350c97e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:44.404744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e4f89b-deea-44ec-be75-261cd741b621 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.415756 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 880/1722] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:44.424046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab11659-4952-430d-8adb-8528477bd47a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.424530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab11659-4952-430d-8adb-8528477bd47a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.424753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab11659-4952-430d-8adb-8528477bd47a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.444885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3ab11659-4952-430d-8adb-8528477bd47a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:44.446084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3ab11659-4952-430d-8adb-8528477bd47a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:44.454325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.454933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.455206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.455533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.457073 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 881/1723] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:44.461311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab11659-4952-430d-8adb-8528477bd47a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.465112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.466667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.470772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab11659-4952-430d-8adb-8528477bd47a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.475424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.476019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.476136 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 843/1724] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:44.476370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.476601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.477247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:44.477618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.479216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.493055 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 882/1725] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:56:44.496829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.507528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e9de9b-3c3c-4259-a613-a0ea90f26ed3 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.512806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.513404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.513679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.513945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.514609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.516516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.545820 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 844/1726] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:44 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:44.549912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.563111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd7be1f-b744-49e3-a869-459f5550eb2a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.620369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.621023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.621341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.621710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.622454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.625058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.652899 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 883/1727] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:44 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:44.657203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.667813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7746d0-7b9d-47ad-84b2-353c1017aa27 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.673068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.673672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.673948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.674218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.674855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.676455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.687518 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 845/1728] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:44.691647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.708405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05892d72-6df0-4d5c-990f-545512c8b047 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.713240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b27dfd-4480-4d98-bd52-d45d7ae4de3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.713994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b27dfd-4480-4d98-bd52-d45d7ae4de3f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.714279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b27dfd-4480-4d98-bd52-d45d7ae4de3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.714560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b27dfd-4480-4d98-bd52-d45d7ae4de3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.715218 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b27dfd-4480-4d98-bd52-d45d7ae4de3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.717028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b27dfd-4480-4d98-bd52-d45d7ae4de3f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.732271 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 884/1729] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:44.736988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.750104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94a0957-84cc-4883-a814-5dc3d96af170 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.756749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.757481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.757760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.758031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] PATH_INFO: `/v3/projects/f64606c140ba4bc89421d430c05d81ee/users/3e6fced3913f41eabc96b63a6651a2e6/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.776113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64606c140ba4bc89421d430c05d81ee, user_id=3e6fced3913f41eabc96b63a6651a2e6, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.778958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.796721 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 846/1730] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:44 2022] PUT /identity/v3/projects/f64606c140ba4bc89421d430c05d81ee/users/3e6fced3913f41eabc96b63a6651a2e6/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:44.807373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6768d-491b-40c8-bb1b-cb1a2be5ad99 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.807674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6768d-491b-40c8-bb1b-cb1a2be5ad99 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.807884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6768d-491b-40c8-bb1b-cb1a2be5ad99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.837892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.839840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:44.846363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:44.853223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-28d6768d-491b-40c8-bb1b-cb1a2be5ad99 None None] MFA Rules not processed for user `3e6fced3913f41eabc96b63a6651a2e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:44.854698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-28d6768d-491b-40c8-bb1b-cb1a2be5ad99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:44.926157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:44.942664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1740-0dd1-4d3b-a95d-340992494eb9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.945809 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 885/1731] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:44.947370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.948001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.948278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.948578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] PATH_INFO: `/v3/projects/0d752c4b92b6429382c11751ad11b422` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.949805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d752c4b92b6429382c11751ad11b422)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.950318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6768d-491b-40c8-bb1b-cb1a2be5ad99 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:44.952292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.961610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6768d-491b-40c8-bb1b-cb1a2be5ad99 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:44.966107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:44.966697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:44.966973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:44.967768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:44.968412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:44.970289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:44.997972 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 886/1732] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:44 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:45.001829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.003069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d752c4b92b6429382c11751ad11b422', 'request_id': 'req-087b47d1-daf4-45ee-81a2-54c32d7ca894'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:45.010260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d752c4b92b6429382c11751ad11b422', 'request_id': 'req-087b47d1-daf4-45ee-81a2-54c32d7ca894'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:45.016205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d752c4b92b6429382c11751ad11b422', 'request_id': 'req-087b47d1-daf4-45ee-81a2-54c32d7ca894'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:45.029389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] The token cache is being invalidate because project 0d752c4b92b6429382c11751ad11b422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:45.030222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:45.031278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-087b47d1-daf4-45ee-81a2-54c32d7ca894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:45.032957 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 847/1733] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:44 2022] DELETE /identity/v3/projects/0d752c4b92b6429382c11751ad11b422 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:45.037616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60e793-785d-4a3a-8470-d12be4b217cd None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.037893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60e793-785d-4a3a-8470-d12be4b217cd None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.038157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60e793-785d-4a3a-8470-d12be4b217cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.057399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-be60e793-785d-4a3a-8470-d12be4b217cd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:45.058604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-be60e793-785d-4a3a-8470-d12be4b217cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:45.064279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8932bcdb-5bff-4675-b7b5-a031dcd6e9d4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.066961 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 848/1734] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:45.068847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.069446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.069722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.070052 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.070618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.072535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.074261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7b2ff-18d4-4740-b3bf-c2d24b839aef None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.074613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7b2ff-18d4-4740-b3bf-c2d24b839aef None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.074955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7b2ff-18d4-4740-b3bf-c2d24b839aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.096608 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 887/1735] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:45 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:45.100048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.101696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:45.105748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c2c7b2ff-18d4-4740-b3bf-c2d24b839aef None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:45.106917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c2c7b2ff-18d4-4740-b3bf-c2d24b839aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:45.114368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1867518b-0bd3-468d-b37e-238f6a0add3a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.115381 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 849/1736] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:45.119575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7b2ff-18d4-4740-b3bf-c2d24b839aef None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.122031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.122639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.122938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.123211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.123912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.125561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.129033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7b2ff-18d4-4740-b3bf-c2d24b839aef None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.133637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.134226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.134503 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.134884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.135523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:45.139581 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 888/1737] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:45.139721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.139721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.140374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.149823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c55f59-c7b2-492f-8b5a-4bed4cf7e573 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.149970 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 850/1738] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:56:45.153994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.154489 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.155718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.155993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.156263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] PATH_INFO: `/v3/projects/0cc0686e3d9b4a8f82d1a06db51b1735/users/4d34829f60104887b8edd74d4f673dd1/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.164058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee464b4-51d2-4b21-8074-b8239424905b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.169446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.171314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.171919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.172035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc0686e3d9b4a8f82d1a06db51b1735, user_id=4d34829f60104887b8edd74d4f673dd1, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.172474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.173246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.174890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.175561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.188615 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 889/1739] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:45 2022] PUT /identity/v3/projects/0cc0686e3d9b4a8f82d1a06db51b1735/users/4d34829f60104887b8edd74d4f673dd1/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:45.192159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.209443 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 851/1740] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:45 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:45.213799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.265508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3971dbb-2d0e-4772-be1a-936ead0ccf25 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.276096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:45.301507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502dfd1-c88b-4225-85d0-6eb58387b5b8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.301810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:45.307715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.308087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.308494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.308859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.309955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.313176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.341727 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 852/1741] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:45 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:45.344868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.345599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.345933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.346209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.347046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.348708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.349358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.364609 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 890/1742] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:45.369420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.377226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe55a90-1ad1-492a-8627-52e5db362fbe admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.382047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a122a1-8151-4b60-85e1-e26f71f37cad admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.390496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.391340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.391693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.391823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbccab-35be-463e-9313-5ed1eb1a6f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.392097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] PATH_INFO: `/v3/projects/0cc0686e3d9b4a8f82d1a06db51b1735/users/4d34829f60104887b8edd74d4f673dd1/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.392949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbccab-35be-463e-9313-5ed1eb1a6f4e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.393294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbccab-35be-463e-9313-5ed1eb1a6f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.393570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbccab-35be-463e-9313-5ed1eb1a6f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.394238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbccab-35be-463e-9313-5ed1eb1a6f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.396501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cc0686e3d9b4a8f82d1a06db51b1735, user_id=4d34829f60104887b8edd74d4f673dd1, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.396677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbccab-35be-463e-9313-5ed1eb1a6f4e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.399655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.412721 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 891/1743] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:45 2022] PUT /identity/v3/projects/0cc0686e3d9b4a8f82d1a06db51b1735/users/4d34829f60104887b8edd74d4f673dd1/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:45.414645 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 853/1744] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:45.417751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.425695 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.426302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.426680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.469620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] MFA Rules not processed for user `4d34829f60104887b8edd74d4f673dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:45.471110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:45.509182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1355d1-311b-4047-a59b-51355c5f59d7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.515458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.516246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.516651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.517128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.517851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.519539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.531249 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 892/1745] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:45.535240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.546777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d96a42-f640-48f9-93cc-24c41f097b59 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.551832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aea991-f651-455c-8563-3c172829842f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.552689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aea991-f651-455c-8563-3c172829842f admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.553134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aea991-f651-455c-8563-3c172829842f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.553370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aea991-f651-455c-8563-3c172829842f admin admin] PATH_INFO: `/v3/projects/370e8dd001694f989ed4360a0d629c36/users/7b4c3314327c4bdfa78a26c9fc827f2c/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.562740 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 854/1746] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:45.570707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aea991-f651-455c-8563-3c172829842f admin admin] RBAC: Authorizing `identity:create_grant(project_id=370e8dd001694f989ed4360a0d629c36, user_id=7b4c3314327c4bdfa78a26c9fc827f2c, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.573544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aea991-f651-455c-8563-3c172829842f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.574216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.575714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:45.613863 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 893/1747] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:45 2022] PUT /identity/v3/projects/370e8dd001694f989ed4360a0d629c36/users/7b4c3314327c4bdfa78a26c9fc827f2c/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:45.624667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.625417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.625417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.661244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] MFA Rules not processed for user `7b4c3314327c4bdfa78a26c9fc827f2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:45.663185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:45.680062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38966998-4d0c-4156-984b-ef5f0a2b6ff4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.738204 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 894/1748] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:45.741481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f60ce2-1916-4504-8ec8-8dcb50a7d175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.742730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f60ce2-1916-4504-8ec8-8dcb50a7d175 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.743011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f60ce2-1916-4504-8ec8-8dcb50a7d175 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.743277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f60ce2-1916-4504-8ec8-8dcb50a7d175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.749667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:45.750408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:45.766025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b77343a-4778-4e7d-9ed6-e909b0a455bd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:45.770330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:45.771003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:45.771315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:45.771534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:45.771867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:45.782702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.784526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.794808 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 895/1749] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:45.836439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f60ce2-1916-4504-8ec8-8dcb50a7d175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:45.836858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f60ce2-1916-4504-8ec8-8dcb50a7d175 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:45.850228 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 855/1750] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:47.472378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:47.472378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:47.539128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04508dff-1e1e-4e1f-877d-5407e72b32a2 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:47.544694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:47.545810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:47.546084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:47.546358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] PATH_INFO: `/v3/users/3e6fced3913f41eabc96b63a6651a2e6` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:47.547484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6fced3913f41eabc96b63a6651a2e6)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:47.549435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:47.614529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6fced3913f41eabc96b63a6651a2e6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:47.622491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6fced3913f41eabc96b63a6651a2e6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:47.632089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6fced3913f41eabc96b63a6651a2e6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:47.647641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6fced3913f41eabc96b63a6651a2e6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:47.648666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6fced3913f41eabc96b63a6651a2e6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:47.656346 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 896/1751] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:47 2022] DELETE /identity/v3/users/3e6fced3913f41eabc96b63a6651a2e6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:47.660088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:47.660570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:47.660754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:47.682678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:47.683935 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:47.772157 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 856/1752] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:47.782379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:47.783888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:47.844796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eec299-4725-4fed-9492-601b0811ba42 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:47.848485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:47.849124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:47.849440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:47.849676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:47.866264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:47.867695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:47.878223 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 897/1753] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:48.277015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:48.278581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:48.293799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165b383-1ed9-42ea-bfad-841dafabe2dc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:48.298981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:48.299627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:48.299907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:48.300181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] PATH_INFO: `/v3/projects/f64606c140ba4bc89421d430c05d81ee` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:48.301369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64606c140ba4bc89421d430c05d81ee)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:48.303271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:48.356425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64606c140ba4bc89421d430c05d81ee', 'request_id': 'req-455737a5-75d2-4a36-8aad-80716721738e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.368383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64606c140ba4bc89421d430c05d81ee', 'request_id': 'req-455737a5-75d2-4a36-8aad-80716721738e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.378713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64606c140ba4bc89421d430c05d81ee', 'request_id': 'req-455737a5-75d2-4a36-8aad-80716721738e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.394520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] The token cache is being invalidate because project f64606c140ba4bc89421d430c05d81ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:48.394805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.395473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.397773 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 857/1754] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:48 2022] DELETE /identity/v3/projects/f64606c140ba4bc89421d430c05d81ee => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:48.401517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:48.412641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdc17e-9103-4cf1-90aa-f3f5102785b8 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:48.436435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:48.500001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:48.500693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:48.500975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:48.501249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] PATH_INFO: `/v3/users/4d34829f60104887b8edd74d4f673dd1` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:48.502409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d34829f60104887b8edd74d4f673dd1)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:48.504690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:48.552598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d34829f60104887b8edd74d4f673dd1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.559657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d34829f60104887b8edd74d4f673dd1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.566908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d34829f60104887b8edd74d4f673dd1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.578971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d34829f60104887b8edd74d4f673dd1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.580084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d34829f60104887b8edd74d4f673dd1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:48.585869 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 898/1755] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:48 2022] DELETE /identity/v3/users/4d34829f60104887b8edd74d4f673dd1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:48.949048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:48.950520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:48.966050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455737a5-75d2-4a36-8aad-80716721738e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.005096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.007459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.024121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:49.048577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.049324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.049609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.049879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] PATH_INFO: `/v3/projects/0cc0686e3d9b4a8f82d1a06db51b1735` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.051103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cc0686e3d9b4a8f82d1a06db51b1735)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.053071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.100815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ef6c43-6310-4952-9719-d0128f632eae admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.109171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cc0686e3d9b4a8f82d1a06db51b1735', 'request_id': 'req-6f10dea9-e194-46ba-8801-1a29842c5f27'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.118167 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cc0686e3d9b4a8f82d1a06db51b1735', 'request_id': 'req-6f10dea9-e194-46ba-8801-1a29842c5f27'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.126943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cc0686e3d9b4a8f82d1a06db51b1735', 'request_id': 'req-6f10dea9-e194-46ba-8801-1a29842c5f27'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.143656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] The token cache is being invalidate because project 0cc0686e3d9b4a8f82d1a06db51b1735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:49.143950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.144687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-6f10dea9-e194-46ba-8801-1a29842c5f27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.146461 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 858/1756] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:48 2022] DELETE /identity/v3/projects/0cc0686e3d9b4a8f82d1a06db51b1735 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:49.151067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8eb46-e297-4321-9520-7743bfdd3e64 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.151446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8eb46-e297-4321-9520-7743bfdd3e64 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.151811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8eb46-e297-4321-9520-7743bfdd3e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.163194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:49.174405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a0b8eb46-e297-4321-9520-7743bfdd3e64 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:49.176078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a0b8eb46-e297-4321-9520-7743bfdd3e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.176256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.177344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.177843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.178178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] PATH_INFO: `/v3/users/7b4c3314327c4bdfa78a26c9fc827f2c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.179363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b4c3314327c4bdfa78a26c9fc827f2c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.181439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.185427 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 859/1757] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:49.193154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411596d-10c7-4e4c-b13f-a727fbf3144f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.193551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411596d-10c7-4e4c-b13f-a727fbf3144f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.193916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411596d-10c7-4e4c-b13f-a727fbf3144f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.219796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2411596d-10c7-4e4c-b13f-a727fbf3144f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:49.221169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2411596d-10c7-4e4c-b13f-a727fbf3144f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.229945 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 860/1758] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:49.234990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411596d-10c7-4e4c-b13f-a727fbf3144f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.236784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b4c3314327c4bdfa78a26c9fc827f2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.246566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b4c3314327c4bdfa78a26c9fc827f2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.249162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411596d-10c7-4e4c-b13f-a727fbf3144f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.258355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.258488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b4c3314327c4bdfa78a26c9fc827f2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.259158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.259442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.259713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.260772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:49.261704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.263566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.280470 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 861/1759] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:56:49.285431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.286366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b4c3314327c4bdfa78a26c9fc827f2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.300636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-779103d0-1606-47ca-8537-e88c86f7750b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b4c3314327c4bdfa78a26c9fc827f2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.300636 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 899/1760] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:49 2022] DELETE /identity/v3/users/7b4c3314327c4bdfa78a26c9fc827f2c => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:49.300636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.300636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.300636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.306800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d598a-2db2-42a4-b403-d180f0a18c5b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.320984 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:49.322266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.369820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.371093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.373879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.373879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.374415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.376655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.407104 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 862/1761] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/projects => generated 407 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:49.408494 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 900/1762] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:49.414144 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.415930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.418663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.420110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.450854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:49.494825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c7592-e235-40ad-8a6d-7b3816f7dbc6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.497025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09558b-e06c-4509-bf76-3a3c64d8d5dd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.499998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.500842 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.501125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.501395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.502054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.504230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.523333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:49.531458 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 863/1763] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/users => generated 304 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:49.532605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.543921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f95e4-de96-4612-9a86-d994bf2a8657 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.550559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.551746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.552163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.552672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.553032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.553476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.553656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.553918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.554191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.555602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.565499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.566933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.567766 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 864/1764] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:49.571743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.578162 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 901/1765] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:49.581481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0797426-1cda-4711-b1f4-b0b6cf44ecb7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.586127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9b21ea-1a53-4c2d-883d-57c9382afc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.586759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b21ea-1a53-4c2d-883d-57c9382afc16 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.587046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b21ea-1a53-4c2d-883d-57c9382afc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.587311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9b21ea-1a53-4c2d-883d-57c9382afc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.587947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b21ea-1a53-4c2d-883d-57c9382afc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.589601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9b21ea-1a53-4c2d-883d-57c9382afc16 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.602177 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 865/1766] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:49.606500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.617487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928e896-f4f4-4726-8ce0-e1961656266f service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.622144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.622801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.623078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.623400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] PATH_INFO: `/v3/projects/bd4d7089e3d64e74b3bf6220c4e2643a/users/f377ac896a72490d8a2a53b8c7e79ddc/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.640771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd4d7089e3d64e74b3bf6220c4e2643a, user_id=f377ac896a72490d8a2a53b8c7e79ddc, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.643571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.657598 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 902/1767] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:49 2022] PUT /identity/v3/projects/bd4d7089e3d64e74b3bf6220c4e2643a/users/f377ac896a72490d8a2a53b8c7e79ddc/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:49.669412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.669696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.669970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.701279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] MFA Rules not processed for user `f377ac896a72490d8a2a53b8c7e79ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:49.703092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.793807 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 866/1768] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:49.798876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.867796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.869319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:49.878807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e83be-def6-44a8-91ce-8c8ff635c0d5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.885642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.887161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68236098-a026-4fbd-a9ce-961885846df5 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:49.887939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.888243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.888550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.889630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.891505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.894199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:49.894846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:49.895139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:49.895401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] PATH_INFO: `/v3/projects/370e8dd001694f989ed4360a0d629c36` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:49.896567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] RBAC: Authorizing `identity:delete_project(project_id=370e8dd001694f989ed4360a0d629c36)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:49.898784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:49.923948 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 903/1769] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/projects => generated 407 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:49.928142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:49.961299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '370e8dd001694f989ed4360a0d629c36', 'request_id': 'req-eb940448-1888-429d-bacb-f6f1cc0a296d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.973425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '370e8dd001694f989ed4360a0d629c36', 'request_id': 'req-eb940448-1888-429d-bacb-f6f1cc0a296d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:49.983430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '370e8dd001694f989ed4360a0d629c36', 'request_id': 'req-eb940448-1888-429d-bacb-f6f1cc0a296d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:50.000642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] The token cache is being invalidate because project 370e8dd001694f989ed4360a0d629c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:50.001401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:50.001870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4554d75-e71f-43b9-97f0-d3dfa0bbbdc9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:50.002480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:50.004600 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 867/1770] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:49 2022] DELETE /identity/v3/projects/370e8dd001694f989ed4360a0d629c36 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:50.058325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:50.092821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:50.098621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.099805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.100238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.101153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:50.103348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:50.149782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:50.150451 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 904/1771] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:49 2022] POST /identity/v3/users => generated 310 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:50.152489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:50.170826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb940448-1888-429d-bacb-f6f1cc0a296d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:50.180869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:50.181840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.182482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.182928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.183735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:50.190188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:50.193287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:50.204366 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 868/1772] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:50.208971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:50.219954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c422aa0-0c45-4585-ac81-dd60ff0150e0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:50.224646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:50.225300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.225576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.225847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] PATH_INFO: `/v3/projects/6aef111871b14988b60ee124f50ab0e0/users/7bceafae38aa48c486e62e2ca9a91127/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.240858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aef111871b14988b60ee124f50ab0e0, user_id=7bceafae38aa48c486e62e2ca9a91127, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:50.243589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:50.257706 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 905/1773] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:50 2022] PUT /identity/v3/projects/6aef111871b14988b60ee124f50ab0e0/users/7bceafae38aa48c486e62e2ca9a91127/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:56:50.261742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:50.272166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b00661b-3004-4716-a428-64aa6e8946c7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:50.346507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92155756-26b5-4a8c-ac11-a6b20d606263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:50.347127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155756-26b5-4a8c-ac11-a6b20d606263 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.347408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155756-26b5-4a8c-ac11-a6b20d606263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.347679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92155756-26b5-4a8c-ac11-a6b20d606263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.348342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92155756-26b5-4a8c-ac11-a6b20d606263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:50.349912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92155756-26b5-4a8c-ac11-a6b20d606263 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:50.359904 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 869/1774] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:50.364322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:50.375178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54691c34-d1f5-4e87-9d2c-28f4f4544d02 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:50.380003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:50.380786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.381130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.381402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] PATH_INFO: `/v3/projects/6aef111871b14988b60ee124f50ab0e0/users/7bceafae38aa48c486e62e2ca9a91127/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.383266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aef111871b14988b60ee124f50ab0e0, user_id=7bceafae38aa48c486e62e2ca9a91127, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:50.385933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:50.397298 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 906/1775] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:50 2022] PUT /identity/v3/projects/6aef111871b14988b60ee124f50ab0e0/users/7bceafae38aa48c486e62e2ca9a91127/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:50.407672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa66e7-3ab8-4671-94bd-89a500820ba4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.407956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa66e7-3ab8-4671-94bd-89a500820ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.408229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa66e7-3ab8-4671-94bd-89a500820ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.437906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b2fa66e7-3ab8-4671-94bd-89a500820ba4 None None] MFA Rules not processed for user `7bceafae38aa48c486e62e2ca9a91127`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:50.439160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b2fa66e7-3ab8-4671-94bd-89a500820ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:50.507575 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 870/1776] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:50 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:50.518049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:50.519715 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:50.578148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c537ba9b-f6bd-4c04-b0ff-436a872d81fd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:50.581967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:50.582572 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.582851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.583120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.591991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:50.593448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:50.602588 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 907/1777] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:50 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:50.757129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa66e7-3ab8-4671-94bd-89a500820ba4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:50.769677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa66e7-3ab8-4671-94bd-89a500820ba4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:50.773202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad3930d-38c1-4ec5-ae81-0cf5a3a8f381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:50.773804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3930d-38c1-4ec5-ae81-0cf5a3a8f381 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:50.774084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3930d-38c1-4ec5-ae81-0cf5a3a8f381 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:50.774356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad3930d-38c1-4ec5-ae81-0cf5a3a8f381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:50.776067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4ad3930d-38c1-4ec5-ae81-0cf5a3a8f381 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:50.834724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3930d-38c1-4ec5-ae81-0cf5a3a8f381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:50.836145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad3930d-38c1-4ec5-ae81-0cf5a3a8f381 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:50.845629 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 871/1778] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:50 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:54.264364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:54.266343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:54.277158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a89dc2-8eaf-4745-86f8-e8a46012670d service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:54.281455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:54.282047 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:54.282325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:54.282598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:54.284275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:54.345548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:54.347038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:54.356564 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 908/1779] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:54.886524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049a41-c128-4c4c-820c-0b8d82867b62 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:54.887019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049a41-c128-4c4c-820c-0b8d82867b62 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:54.887203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf049a41-c128-4c4c-820c-0b8d82867b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:54.908742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-cf049a41-c128-4c4c-820c-0b8d82867b62 None None] MFA Rules not processed for user `08616a72155d4b8595bb7d813b155ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:54.909879 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-cf049a41-c128-4c4c-820c-0b8d82867b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:54.921945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:54.971044 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 872/1780] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:56.248169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:56.278296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:56.317649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a67e6-a066-4ed7-8246-408435e77ce3 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:56.322328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:56.323726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:56.324015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:56.324483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] PATH_INFO: `/v3/users/f377ac896a72490d8a2a53b8c7e79ddc` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:56.325662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f377ac896a72490d8a2a53b8c7e79ddc)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:56.328291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:56.380821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f377ac896a72490d8a2a53b8c7e79ddc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:56.388583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f377ac896a72490d8a2a53b8c7e79ddc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:56.398337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f377ac896a72490d8a2a53b8c7e79ddc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:56.410511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f377ac896a72490d8a2a53b8c7e79ddc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:56.411217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f377ac896a72490d8a2a53b8c7e79ddc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:56.417301 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 909/1781] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:56 2022] DELETE /identity/v3/users/f377ac896a72490d8a2a53b8c7e79ddc => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:56.422219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:56.423314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:56.423799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:56.444947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:56.446161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:56.512576 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 873/1782] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:56.523566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:56.525007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:56.587540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d72cc5-b6b5-4f64-9bd8-f44d6b2835a8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:56.591156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:56.591764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:56.592068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:56.592399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:56.601618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:56.603079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:56.616370 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 910/1783] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:56:57.020160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:57.021761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:57.035661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1b573-e4df-4631-b259-2f10887b180b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:57.041453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:57.042289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:57.042746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:57.043189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] PATH_INFO: `/v3/projects/bd4d7089e3d64e74b3bf6220c4e2643a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:57.044709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd4d7089e3d64e74b3bf6220c4e2643a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:57.047397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:57.101603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd4d7089e3d64e74b3bf6220c4e2643a', 'request_id': 'req-e903e8bf-103e-4264-8413-9a4238ebedf7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.109937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd4d7089e3d64e74b3bf6220c4e2643a', 'request_id': 'req-e903e8bf-103e-4264-8413-9a4238ebedf7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.116034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd4d7089e3d64e74b3bf6220c4e2643a', 'request_id': 'req-e903e8bf-103e-4264-8413-9a4238ebedf7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.127294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] The token cache is being invalidate because project bd4d7089e3d64e74b3bf6220c4e2643a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:57.127596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.128547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.130170 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 874/1784] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:57 2022] DELETE /identity/v3/projects/bd4d7089e3d64e74b3bf6220c4e2643a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:57.134173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:57.145108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7b5ab3-dc2a-4fd9-b14c-369f2b0b8ad4 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:57.191730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:57.206308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:57.207037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:57.207319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:57.207601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] PATH_INFO: `/v3/users/7bceafae38aa48c486e62e2ca9a91127` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:57.208858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bceafae38aa48c486e62e2ca9a91127)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:57.210831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:57.257746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bceafae38aa48c486e62e2ca9a91127'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.265599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bceafae38aa48c486e62e2ca9a91127'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.274507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bceafae38aa48c486e62e2ca9a91127'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.288851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bceafae38aa48c486e62e2ca9a91127'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.289579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d8446306-5a27-46f8-bdd2-581f17ea96ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bceafae38aa48c486e62e2ca9a91127'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.296681 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 911/1785] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:57 2022] DELETE /identity/v3/users/7bceafae38aa48c486e62e2ca9a91127 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:57.644149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:57.648134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:57.687764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e903e8bf-103e-4264-8413-9a4238ebedf7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:57.742351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:57.762295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:57.763405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:57.766131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:57.766439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] PATH_INFO: `/v3/projects/6aef111871b14988b60ee124f50ab0e0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:57.770055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aef111871b14988b60ee124f50ab0e0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:57.772771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:57.824694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aef111871b14988b60ee124f50ab0e0', 'request_id': 'req-9ee67c75-6280-4481-8497-133b7c5b90c9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.843715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aef111871b14988b60ee124f50ab0e0', 'request_id': 'req-9ee67c75-6280-4481-8497-133b7c5b90c9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.850286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aef111871b14988b60ee124f50ab0e0', 'request_id': 'req-9ee67c75-6280-4481-8497-133b7c5b90c9'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.863368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] The token cache is being invalidate because project 6aef111871b14988b60ee124f50ab0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:57.863693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.864318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-9ee67c75-6280-4481-8497-133b7c5b90c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:57.866687 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 875/1786] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:57 2022] DELETE /identity/v3/projects/6aef111871b14988b60ee124f50ab0e0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:57.871806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0fcb3-a44a-438d-b4d0-1597ff755a13 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:57.872609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0fcb3-a44a-438d-b4d0-1597ff755a13 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:57.872917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b0fcb3-a44a-438d-b4d0-1597ff755a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:57.895071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e1b0fcb3-a44a-438d-b4d0-1597ff755a13 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:57.896588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e1b0fcb3-a44a-438d-b4d0-1597ff755a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:57.966480 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 912/1787] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:57.974814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1a25c-13c4-4262-9441-079ab5edf4ed None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:57.975117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1a25c-13c4-4262-9441-079ab5edf4ed None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:57.975387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1a25c-13c4-4262-9441-079ab5edf4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:57.996618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0af1a25c-13c4-4262-9441-079ab5edf4ed None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:57.997907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0af1a25c-13c4-4262-9441-079ab5edf4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:58.011300 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 876/1788] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:58.016400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0fcb3-a44a-438d-b4d0-1597ff755a13 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.027397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b0fcb3-a44a-438d-b4d0-1597ff755a13 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.031993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.032625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.032897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.033163 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.033832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:58.034128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.035833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.049712 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 913/1789] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:58.054026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1a25c-13c4-4262-9441-079ab5edf4ed None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.068134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1a25c-13c4-4262-9441-079ab5edf4ed None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.073006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.073601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.073873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.074189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.074891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.076808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.103653 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 877/1790] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:58 2022] POST /identity/v3/projects => generated 359 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:58.107902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.118301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9763fa-897a-47d8-b6af-78a52ef7449f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.168596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:58.177110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.177825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.178736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.179257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.180502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.183016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.205322 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 914/1791] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:58 2022] POST /identity/v3/users => generated 280 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:58.209601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.220410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cbbdd-6aba-428a-a7e9-4811f93f69c8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.225440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.226497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.226497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.226738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.227389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.229075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.241408 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 878/1792] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:58.245348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.257775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29231cc-27f3-49f2-b6d7-19293a814607 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.265194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058abfdf-aa60-44a7-b64a-884730e89d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.266166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058abfdf-aa60-44a7-b64a-884730e89d08 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.266735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058abfdf-aa60-44a7-b64a-884730e89d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.267223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058abfdf-aa60-44a7-b64a-884730e89d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.267985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058abfdf-aa60-44a7-b64a-884730e89d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.269936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058abfdf-aa60-44a7-b64a-884730e89d08 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.295134 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 915/1793] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:58 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:58.300276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.313207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ce295-9c25-47fb-a695-96a452d6630c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.318186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.319119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.319516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.319859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] PATH_INFO: `/v3/projects/e112eaaf5eb448ea9061846930f68b9c/users/0c49e1247d8b42f8b85ba4bd695d8720/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.345445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e112eaaf5eb448ea9061846930f68b9c, user_id=0c49e1247d8b42f8b85ba4bd695d8720, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.349128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.364723 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 879/1794] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:58 2022] PUT /identity/v3/projects/e112eaaf5eb448ea9061846930f68b9c/users/0c49e1247d8b42f8b85ba4bd695d8720/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:58.376248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6015f1d-bc25-4d21-ae86-7d441da107eb None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.376560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6015f1d-bc25-4d21-ae86-7d441da107eb None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.376856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6015f1d-bc25-4d21-ae86-7d441da107eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.410361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e6015f1d-bc25-4d21-ae86-7d441da107eb None None] MFA Rules not processed for user `0c49e1247d8b42f8b85ba4bd695d8720`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:58.411671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e6015f1d-bc25-4d21-ae86-7d441da107eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:58.476876 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 916/1795] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:58 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:58.512113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.513930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:58.574792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd4614-1848-4605-b103-e34018e7879a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.578473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd7076-3356-4bc4-9d9c-dcbfbc0ab206 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.579082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd7076-3356-4bc4-9d9c-dcbfbc0ab206 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.579364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd7076-3356-4bc4-9d9c-dcbfbc0ab206 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.579636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd7076-3356-4bc4-9d9c-dcbfbc0ab206 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.588175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd7076-3356-4bc4-9d9c-dcbfbc0ab206 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.589631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dd7076-3356-4bc4-9d9c-dcbfbc0ab206 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.599325 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 880/1796] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:58 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:58.614845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6015f1d-bc25-4d21-ae86-7d441da107eb None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.687724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6015f1d-bc25-4d21-ae86-7d441da107eb None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:58.691912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:58.692831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.693173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.693540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] PATH_INFO: `/v3/users/0c49e1247d8b42f8b85ba4bd695d8720` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.694903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c49e1247d8b42f8b85ba4bd695d8720)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:58.696944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:58.759558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c49e1247d8b42f8b85ba4bd695d8720'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:58.768735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c49e1247d8b42f8b85ba4bd695d8720'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:58.779774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c49e1247d8b42f8b85ba4bd695d8720'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:58.785485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:58.806110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c49e1247d8b42f8b85ba4bd695d8720'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:58.807139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c49e1247d8b42f8b85ba4bd695d8720'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:58.821998 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 917/1797] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:56:58 2022] DELETE /identity/v3/users/0c49e1247d8b42f8b85ba4bd695d8720 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:58.826820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:58.827877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:58.828227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:58.855589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:58.856963 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:58.867529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:58.931024 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 881/1798] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:58.943314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:58.945095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:59.059409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a0c62-966f-44a8-9b6a-0829eddf2ff8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.064476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b620260-2b75-47e7-9190-c14417010866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.065383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b620260-2b75-47e7-9190-c14417010866 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.065496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b620260-2b75-47e7-9190-c14417010866 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.065708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b620260-2b75-47e7-9190-c14417010866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.094789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b620260-2b75-47e7-9190-c14417010866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.096561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b620260-2b75-47e7-9190-c14417010866 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.107210 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 918/1799] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:56:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:56:59.391551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:59.393077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:59.405370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336786c3-2513-46f6-b4f2-4e9408d48721 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.410105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.410731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.411009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.411277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] PATH_INFO: `/v3/projects/e112eaaf5eb448ea9061846930f68b9c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.412455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e112eaaf5eb448ea9061846930f68b9c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.414409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.488561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e112eaaf5eb448ea9061846930f68b9c', 'request_id': 'req-08d5f753-952a-4f98-889d-9997cda44d12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:59.497637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e112eaaf5eb448ea9061846930f68b9c', 'request_id': 'req-08d5f753-952a-4f98-889d-9997cda44d12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:59.504532 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e112eaaf5eb448ea9061846930f68b9c', 'request_id': 'req-08d5f753-952a-4f98-889d-9997cda44d12'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:59.518706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] The token cache is being invalidate because project e112eaaf5eb448ea9061846930f68b9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:56:59.518999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:59.519701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-08d5f753-952a-4f98-889d-9997cda44d12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:56:59.521259 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 882/1800] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:56:59 2022] DELETE /identity/v3/projects/e112eaaf5eb448ea9061846930f68b9c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:59.526116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad7d3-9e08-459f-bad0-fc1ac710e4f0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.526589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad7d3-9e08-459f-bad0-fc1ac710e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.526681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ad7d3-9e08-459f-bad0-fc1ac710e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.548118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4c0ad7d3-9e08-459f-bad0-fc1ac710e4f0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:59.549482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4c0ad7d3-9e08-459f-bad0-fc1ac710e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:59.613060 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 919/1801] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:56:59.621343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40bf2-07c0-4569-bb06-fa6b4b8e72c1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.621735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40bf2-07c0-4569-bb06-fa6b4b8e72c1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.622036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e40bf2-07c0-4569-bb06-fa6b4b8e72c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.642579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-49e40bf2-07c0-4569-bb06-fa6b4b8e72c1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:59.643710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-49e40bf2-07c0-4569-bb06-fa6b4b8e72c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:56:59.652251 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 883/1802] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:56:59.657079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad7d3-9e08-459f-bad0-fc1ac710e4f0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:59.666663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ad7d3-9e08-459f-bad0-fc1ac710e4f0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.671185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.671785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.672060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.672361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.673007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:56:59.673304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.674895 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.686970 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 920/1803] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:56:59.691510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40bf2-07c0-4569-bb06-fa6b4b8e72c1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:59.701087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e40bf2-07c0-4569-bb06-fa6b4b8e72c1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.706863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.707476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.707900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.708171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.708854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.710739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.739151 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 884/1804] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:56:59 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:56:59.743460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:59.754191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b85a1-3498-4adc-8647-17237ca0b526 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.810222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.810840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.811118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.811937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.812612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.814541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.840636 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 921/1805] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:56:59 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:56:59.841751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:59.852126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4920105d-8ec1-4a34-bf17-505e6885c7fb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.856836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.857535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.857878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.858286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.858994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.860741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.872161 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 885/1806] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:56:59.876839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:59.882312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:56:59.887802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6208be2-823c-4c9f-ae2d-6044b3738a45 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.894055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf80f96-038a-4bd6-bae9-4671d64ae949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.894742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80f96-038a-4bd6-bae9-4671d64ae949 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.895309 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80f96-038a-4bd6-bae9-4671d64ae949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.895764 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf80f96-038a-4bd6-bae9-4671d64ae949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.896636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf80f96-038a-4bd6-bae9-4671d64ae949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.898256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf80f96-038a-4bd6-bae9-4671d64ae949 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.908137 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 922/1807] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:56:59.912255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:56:59.922037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba905170-8205-45cb-aee8-3e70332109ad admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:56:59.926560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:56:59.927200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.927473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.927738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] PATH_INFO: `/v3/projects/7f39dbf8ef784b65883df1c2a9f68a73/users/d3620695c17847cfb3c2d246b857d5ed/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.942390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f39dbf8ef784b65883df1c2a9f68a73, user_id=d3620695c17847cfb3c2d246b857d5ed, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:56:59.945046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:56:59.957565 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 886/1808] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:56:59 2022] PUT /identity/v3/projects/7f39dbf8ef784b65883df1c2a9f68a73/users/d3620695c17847cfb3c2d246b857d5ed/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:56:59.967616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112e123-c13f-4d2a-a189-91ebfca9f4ea None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:56:59.967901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112e123-c13f-4d2a-a189-91ebfca9f4ea None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:56:59.968173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c112e123-c13f-4d2a-a189-91ebfca9f4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:56:59.996367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c112e123-c13f-4d2a-a189-91ebfca9f4ea None None] MFA Rules not processed for user `d3620695c17847cfb3c2d246b857d5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:56:59.997563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c112e123-c13f-4d2a-a189-91ebfca9f4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:00.060191 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 923/1809] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:56:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:00.065157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:00.111461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:00.131635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50045d47-777b-4119-96fb-5ed6ced47f33 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:00.136346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:00.137057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.137341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.137630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.138298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:00.140792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:00.167808 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 887/1810] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:57:00 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:00.172260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112e123-c13f-4d2a-a189-91ebfca9f4ea None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:00.182891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c112e123-c13f-4d2a-a189-91ebfca9f4ea None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:00.239211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:00.239829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.240107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.240411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.241033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:00.242946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:00.268155 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 924/1811] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:57:00 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:00.269391 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:00.280454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e6d70a-daa6-44d5-9ff3-4914fc8bb85c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:00.285455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:00.286214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.286643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.287110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.287974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:00.289837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:00.304972 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 888/1812] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:00.306154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:00.318698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138aec1c-1fcd-40f3-99db-80dbb798a96f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:00.323650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:00.324328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.324621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.324901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] PATH_INFO: `/v3/projects/5b0fdc38ca834b469a7f5c07c094c1c1/users/32189c2749a64e47941f7dc637c805c2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.340078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0fdc38ca834b469a7f5c07c094c1c1, user_id=32189c2749a64e47941f7dc637c805c2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:00.343017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:00.356363 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 925/1813] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:57:00 2022] PUT /identity/v3/projects/5b0fdc38ca834b469a7f5c07c094c1c1/users/32189c2749a64e47941f7dc637c805c2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:00.361143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:00.372101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582ee169-68ff-4353-89bf-bbefd88ed1a0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:00.440544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c576f1-a187-4090-8755-27fe59923a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:00.441178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c576f1-a187-4090-8755-27fe59923a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.441468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c576f1-a187-4090-8755-27fe59923a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.441729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c576f1-a187-4090-8755-27fe59923a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.442536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c576f1-a187-4090-8755-27fe59923a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:00.444290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c576f1-a187-4090-8755-27fe59923a02 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:00.455190 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 889/1814] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:00 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:00.459844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:00.472600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8bce1-61d3-408b-a5fd-adff6816a7c2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:00.477657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:00.478455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.478728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.479120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] PATH_INFO: `/v3/projects/5b0fdc38ca834b469a7f5c07c094c1c1/users/32189c2749a64e47941f7dc637c805c2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.481215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b0fdc38ca834b469a7f5c07c094c1c1, user_id=32189c2749a64e47941f7dc637c805c2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:00.485134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:00.497130 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 926/1815] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:57:00 2022] PUT /identity/v3/projects/5b0fdc38ca834b469a7f5c07c094c1c1/users/32189c2749a64e47941f7dc637c805c2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:00.510956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed935940-52b4-4adb-b03d-0c2ea1e4e7c2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.511267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed935940-52b4-4adb-b03d-0c2ea1e4e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.511520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed935940-52b4-4adb-b03d-0c2ea1e4e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.541112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ed935940-52b4-4adb-b03d-0c2ea1e4e7c2 None None] MFA Rules not processed for user `32189c2749a64e47941f7dc637c805c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:00.542592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ed935940-52b4-4adb-b03d-0c2ea1e4e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:00.626826 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 890/1816] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:00.637581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:00.639214 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:00.675681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:00.698056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acaac93-ef32-40f3-a0c7-e14938faa3c6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:00.701709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b669dd4d-5363-43da-8300-24a877399f2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:00.702350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669dd4d-5363-43da-8300-24a877399f2c service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:00.702629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669dd4d-5363-43da-8300-24a877399f2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:00.702902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b669dd4d-5363-43da-8300-24a877399f2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:00.757958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669dd4d-5363-43da-8300-24a877399f2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:00.759385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b669dd4d-5363-43da-8300-24a877399f2c service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:00.768256 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 927/1817] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:03.293194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed935940-52b4-4adb-b03d-0c2ea1e4e7c2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:03.326196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:03.358086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed935940-52b4-4adb-b03d-0c2ea1e4e7c2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:03.362511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:03.363148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:03.363423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:03.363693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] PATH_INFO: `/v3/users/d3620695c17847cfb3c2d246b857d5ed` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:03.364844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3620695c17847cfb3c2d246b857d5ed)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:03.366786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:03.415194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3620695c17847cfb3c2d246b857d5ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.415777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3620695c17847cfb3c2d246b857d5ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.422977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3620695c17847cfb3c2d246b857d5ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.434447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3620695c17847cfb3c2d246b857d5ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.435093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3620695c17847cfb3c2d246b857d5ed'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.447675 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 891/1818] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:03 2022] DELETE /identity/v3/users/d3620695c17847cfb3c2d246b857d5ed => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:03.452643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:03.453118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:03.453208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:03.472075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:03.473289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:03.530365 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 928/1819] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:03.540046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:03.541444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:03.599895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3409a-3f3d-425a-89b5-c6125587b9db admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:03.603547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:03.604148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:03.604467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:03.604738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:03.613545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:03.614954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:03.624724 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 892/1820] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:03.881136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:03.882604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:03.895021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f826fe-f8bc-4fa5-bac4-cc83b8931e9b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:03.899745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:03.900421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:03.900790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:03.901042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] PATH_INFO: `/v3/projects/7f39dbf8ef784b65883df1c2a9f68a73` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:03.902709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f39dbf8ef784b65883df1c2a9f68a73)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:03.904917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:03.958009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f39dbf8ef784b65883df1c2a9f68a73', 'request_id': 'req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.965230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f39dbf8ef784b65883df1c2a9f68a73', 'request_id': 'req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.972127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f39dbf8ef784b65883df1c2a9f68a73', 'request_id': 'req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.983418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] The token cache is being invalidate because project 7f39dbf8ef784b65883df1c2a9f68a73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:03.983708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.984370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:03.986164 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 929/1821] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:03 2022] DELETE /identity/v3/projects/7f39dbf8ef784b65883df1c2a9f68a73 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:03.990108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:03.999495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04604ea-0539-4f70-be09-413e170b7bb0 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:04.047191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:04.059785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:04.061129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:04.062684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:04.062960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] PATH_INFO: `/v3/users/32189c2749a64e47941f7dc637c805c2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:04.064149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] RBAC: Authorizing `identity:delete_user(user_id=32189c2749a64e47941f7dc637c805c2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:04.066173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:04.115341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32189c2749a64e47941f7dc637c805c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.116023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32189c2749a64e47941f7dc637c805c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.123138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32189c2749a64e47941f7dc637c805c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.136176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32189c2749a64e47941f7dc637c805c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.136918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-292b2ce7-906c-446d-b253-fe50da22073f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32189c2749a64e47941f7dc637c805c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.153954 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 893/1822] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:03 2022] DELETE /identity/v3/users/32189c2749a64e47941f7dc637c805c2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:57:04.580833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:04.582582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:04.595929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa6ec5-90e9-47d0-8e42-81094a1ef0ef admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:04.603826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:04.665870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:04.666500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:04.666779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:04.667050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] PATH_INFO: `/v3/projects/5b0fdc38ca834b469a7f5c07c094c1c1` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:04.668156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b0fdc38ca834b469a7f5c07c094c1c1)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:04.670097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:04.716554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b0fdc38ca834b469a7f5c07c094c1c1', 'request_id': 'req-6981c0ae-df9d-41c1-9d1a-38168011aebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.723540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b0fdc38ca834b469a7f5c07c094c1c1', 'request_id': 'req-6981c0ae-df9d-41c1-9d1a-38168011aebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.729841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b0fdc38ca834b469a7f5c07c094c1c1', 'request_id': 'req-6981c0ae-df9d-41c1-9d1a-38168011aebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.741526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] The token cache is being invalidate because project 5b0fdc38ca834b469a7f5c07c094c1c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:04.741882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.742507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6981c0ae-df9d-41c1-9d1a-38168011aebf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:04.743970 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 930/1823] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:04 2022] DELETE /identity/v3/projects/5b0fdc38ca834b469a7f5c07c094c1c1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:04.749402 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc0c55-614d-473a-8554-3cae757df165 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:04.749877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc0c55-614d-473a-8554-3cae757df165 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:04.749971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dc0c55-614d-473a-8554-3cae757df165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:04.770126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-50dc0c55-614d-473a-8554-3cae757df165 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:04.771322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-50dc0c55-614d-473a-8554-3cae757df165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:04.830292 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 894/1824] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:04.842127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f50be-5c7d-4d21-98b3-27efdc291c26 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:04.843220 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f50be-5c7d-4d21-98b3-27efdc291c26 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:04.843545 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f50be-5c7d-4d21-98b3-27efdc291c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:04.863711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-ce6f50be-5c7d-4d21-98b3-27efdc291c26 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:04.864909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ce6f50be-5c7d-4d21-98b3-27efdc291c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:04.872549 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 931/1825] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:04.876870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc0c55-614d-473a-8554-3cae757df165 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:04.886912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dc0c55-614d-473a-8554-3cae757df165 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:04.891959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:04.892690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:04.892970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:04.893244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:04.893887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:57:04.894187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:04.895770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:04.907767 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 895/1826] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:57:04.911868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f50be-5c7d-4d21-98b3-27efdc291c26 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:04.921008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f50be-5c7d-4d21-98b3-27efdc291c26 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:04.925615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:04.926208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:04.926485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:04.926755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:04.927383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:04.929303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:04.955206 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 932/1827] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:57:04 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:04.959228 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:04.971375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8531a4-c14b-43e3-a22d-68de2dfa9140 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:05.052702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:05.053328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:05.053607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:05.054210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:05.054512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:05.056850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:05.066000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:05.089336 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 896/1828] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:57:04 2022] POST /identity/v3/users => generated 283 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:05.090591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:05.104519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6fbd3-83f3-44dc-b4b9-2da9b7a8165e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:05.118896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:05.119548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:05.119829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:05.120104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:05.120809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:05.122435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:05.137141 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 933/1829] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:05.138690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:05.150336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe27691-af6f-4322-afa7-4db3ed0af586 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:05.156392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2902d82-a502-410d-803b-c429b033f2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:05.157195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2902d82-a502-410d-803b-c429b033f2ab admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:05.157468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2902d82-a502-410d-803b-c429b033f2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:05.157733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2902d82-a502-410d-803b-c429b033f2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:05.158405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2902d82-a502-410d-803b-c429b033f2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:05.160234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2902d82-a502-410d-803b-c429b033f2ab admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:05.171613 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 897/1830] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:05.175859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:05.186766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fad0e5-5890-416f-92ea-3722c11038a5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:05.191347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:05.191998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:05.192294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:05.192590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] PATH_INFO: `/v3/projects/89b93cbb32fd414c8668488f5dcd0b21/users/4bf78b2498104b05b8135e82a93b4a7b/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:05.207937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] RBAC: Authorizing `identity:create_grant(project_id=89b93cbb32fd414c8668488f5dcd0b21, user_id=4bf78b2498104b05b8135e82a93b4a7b, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:05.210759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:05.224622 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 934/1831] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:57:05 2022] PUT /identity/v3/projects/89b93cbb32fd414c8668488f5dcd0b21/users/4bf78b2498104b05b8135e82a93b4a7b/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:05.235859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23389a7a-1845-4744-a389-314c81a08799 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:05.236202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23389a7a-1845-4744-a389-314c81a08799 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:05.236561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23389a7a-1845-4744-a389-314c81a08799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:05.265859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-23389a7a-1845-4744-a389-314c81a08799 None None] MFA Rules not processed for user `4bf78b2498104b05b8135e82a93b4a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:05.266982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-23389a7a-1845-4744-a389-314c81a08799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:05.331975 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 898/1832] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:05.343084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:05.344950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:05.389420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:05.405888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa83a2-bdcb-4a4b-b2cc-64cb9ee1f33d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:05.409648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:05.410256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:05.410535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:05.410804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:05.419225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:05.421273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:05.430053 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 935/1833] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:14.508609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23389a7a-1845-4744-a389-314c81a08799 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:14.510065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-23389a7a-1845-4744-a389-314c81a08799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:14.582727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23389a7a-1845-4744-a389-314c81a08799 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:14.587316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:14.588092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:14.588242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:14.588533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] PATH_INFO: `/v3/users/58f79424c9154a759bb602f692655bbc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:14.589655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f79424c9154a759bb602f692655bbc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:14.591579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:14.639234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f79424c9154a759bb602f692655bbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:14.639937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f79424c9154a759bb602f692655bbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:14.647054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f79424c9154a759bb602f692655bbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:14.656992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:14.659322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f79424c9154a759bb602f692655bbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:14.660286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f79424c9154a759bb602f692655bbc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:14.674503 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 899/1834] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:14 2022] DELETE /identity/v3/users/58f79424c9154a759bb602f692655bbc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:14.843117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:14.858319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89aae8-4066-4418-bd7a-681ba6afddc8 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:14.923654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:14.924249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:14.924716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:14.924901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] PATH_INFO: `/v3/projects/fbb04f1a5b184c9eb995165da5a3e58a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:14.926327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbb04f1a5b184c9eb995165da5a3e58a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:14.928674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:14.976532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb04f1a5b184c9eb995165da5a3e58a', 'request_id': 'req-254d1eae-403a-4834-82cc-2ada17d85466'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:14.985229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb04f1a5b184c9eb995165da5a3e58a', 'request_id': 'req-254d1eae-403a-4834-82cc-2ada17d85466'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:14.991981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb04f1a5b184c9eb995165da5a3e58a', 'request_id': 'req-254d1eae-403a-4834-82cc-2ada17d85466'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.003311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] The token cache is being invalidate because project fbb04f1a5b184c9eb995165da5a3e58a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:15.003629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.004264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.005800 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 936/1835] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:14 2022] DELETE /identity/v3/projects/fbb04f1a5b184c9eb995165da5a3e58a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:15.009669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.011060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:15.024198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e763f18b-2ce4-433c-8c13-c45560014c11 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.079557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.080221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.080531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.080797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] PATH_INFO: `/v3/users/587f3da8b3534260bb3bc3072bb68983` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.081892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=587f3da8b3534260bb3bc3072bb68983)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.083786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:15.131996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587f3da8b3534260bb3bc3072bb68983'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.132671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587f3da8b3534260bb3bc3072bb68983'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.140006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587f3da8b3534260bb3bc3072bb68983'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.151676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587f3da8b3534260bb3bc3072bb68983'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.152322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5c629d53-49f1-4b62-9a2a-a2a3bc499624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587f3da8b3534260bb3bc3072bb68983'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.165185 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 900/1836] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:15 2022] DELETE /identity/v3/users/587f3da8b3534260bb3bc3072bb68983 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:15.335607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.337063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:15.350615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d1eae-403a-4834-82cc-2ada17d85466 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.415783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.416490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.416779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.417068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] PATH_INFO: `/v3/projects/f8692c5b86f64ab1a98ea2dd6c7cffbe` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.418273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8692c5b86f64ab1a98ea2dd6c7cffbe)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.421124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:15.435884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:15.439099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.439383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.439651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.461054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:15.462276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:15.483410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8692c5b86f64ab1a98ea2dd6c7cffbe', 'request_id': 'req-cc5b24a3-f128-4690-ba53-3641fcceb510'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.492182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8692c5b86f64ab1a98ea2dd6c7cffbe', 'request_id': 'req-cc5b24a3-f128-4690-ba53-3641fcceb510'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.500905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8692c5b86f64ab1a98ea2dd6c7cffbe', 'request_id': 'req-cc5b24a3-f128-4690-ba53-3641fcceb510'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.512666 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] The token cache is being invalidate because project f8692c5b86f64ab1a98ea2dd6c7cffbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:15.512964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.513618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cc5b24a3-f128-4690-ba53-3641fcceb510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:15.515326 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 937/1837] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:15 2022] DELETE /identity/v3/projects/f8692c5b86f64ab1a98ea2dd6c7cffbe => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:15.520277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a99a5-3269-471a-a481-f9d0a3902ce0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.520710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a99a5-3269-471a-a481-f9d0a3902ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.521024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a99a5-3269-471a-a481-f9d0a3902ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.541706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bf3a99a5-3269-471a-a481-f9d0a3902ce0 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:15.542331 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 901/1838] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:15.545906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bf3a99a5-3269-471a-a481-f9d0a3902ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:15.598337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.602837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:15.615250 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 938/1839] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:57:15.622484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:15.624990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c8c2f-24d4-4e9d-a12e-582be593f345 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.625276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c8c2f-24d4-4e9d-a12e-582be593f345 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.625555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c8c2f-24d4-4e9d-a12e-582be593f345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.652647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a69c8c2f-24d4-4e9d-a12e-582be593f345 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:15.653844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a69c8c2f-24d4-4e9d-a12e-582be593f345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:15.662999 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 939/1840] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:15.667127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c8c2f-24d4-4e9d-a12e-582be593f345 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.677262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c8c2f-24d4-4e9d-a12e-582be593f345 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.682119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.682711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.682989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.683258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.683931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:57:15.684230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.685872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:15.697585 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 940/1841] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 00:57:15.702097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.703153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95e60d-05ba-4768-9420-1f1cd02dd90a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.708733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.709978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.710256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.710541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.713402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eecee0-4eb8-4249-b8cd-6214519b1140 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.718829 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.719420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.719701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.719965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.720634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.721488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.722536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:15.723166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:15.731874 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 902/1842] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 00:57:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:57:15.753655 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 941/1843] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:57:15 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:57:15.758211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.791810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a7747-273a-409c-9f9c-8ccc77ce5ec0 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.880015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.880787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.881158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.881548 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.882479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.885344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:15.915180 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 903/1844] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:57:15 2022] POST /identity/v3/users => generated 292 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:15.919405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.933463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a183db7f-8c38-486c-b7db-b7d86ddad11a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.939043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.939653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.939947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.940216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.940930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.942511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:15.957575 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 942/1845] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:15.961299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:15.973501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda357d8-572a-4431-bab2-60ffc940d992 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:15.980006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00427362-e441-4b43-a770-e27dfa0a290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:15.985686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00427362-e441-4b43-a770-e27dfa0a290c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:15.986207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00427362-e441-4b43-a770-e27dfa0a290c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:15.986430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00427362-e441-4b43-a770-e27dfa0a290c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:15.987084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00427362-e441-4b43-a770-e27dfa0a290c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:15.988927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00427362-e441-4b43-a770-e27dfa0a290c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:16.006362 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 904/1846] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:16.007576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:16.019203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70425bb0-b546-4ba1-8d30-ca142df23c4c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:16.025445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:16.026110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:16.026392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:16.026663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] PATH_INFO: `/v3/projects/47712cb5b4834f918d3c4437f5bb0feb/users/a3d313e4587f4c0ba5d74aa38e750e32/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:16.046688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47712cb5b4834f918d3c4437f5bb0feb, user_id=a3d313e4587f4c0ba5d74aa38e750e32, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:16.050256 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:16.070332 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 943/1847] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:57:16 2022] PUT /identity/v3/projects/47712cb5b4834f918d3c4437f5bb0feb/users/a3d313e4587f4c0ba5d74aa38e750e32/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:16.083557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947cfad-b440-4a8e-aa3a-1c47d68cbad0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:16.083998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947cfad-b440-4a8e-aa3a-1c47d68cbad0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:16.086207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947cfad-b440-4a8e-aa3a-1c47d68cbad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:16.119750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4947cfad-b440-4a8e-aa3a-1c47d68cbad0 None None] MFA Rules not processed for user `a3d313e4587f4c0ba5d74aa38e750e32`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:16.121593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4947cfad-b440-4a8e-aa3a-1c47d68cbad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:16.210012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 905/1848] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:16.221655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:16.224106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:16.259179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:16.303688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c0ad0-0f4f-4088-8d3f-effd2528ff71 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:16.308592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aaabb9-8e32-482b-bbc0-bbf531a40ba7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:16.309461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aaabb9-8e32-482b-bbc0-bbf531a40ba7 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:16.309939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aaabb9-8e32-482b-bbc0-bbf531a40ba7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:16.310387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aaabb9-8e32-482b-bbc0-bbf531a40ba7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:16.321531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aaabb9-8e32-482b-bbc0-bbf531a40ba7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:16.323050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67aaabb9-8e32-482b-bbc0-bbf531a40ba7 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:16.335980 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 944/1849] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:16.357455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947cfad-b440-4a8e-aa3a-1c47d68cbad0 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:16.368626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:16.447055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947cfad-b440-4a8e-aa3a-1c47d68cbad0 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:16.452536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:16.453183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:16.453463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:16.453826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] PATH_INFO: `/v3/users/a3d313e4587f4c0ba5d74aa38e750e32` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:16.454995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3d313e4587f4c0ba5d74aa38e750e32)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:16.456952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:16.517658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3d313e4587f4c0ba5d74aa38e750e32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:16.518142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3d313e4587f4c0ba5d74aa38e750e32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:16.544508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3d313e4587f4c0ba5d74aa38e750e32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:16.581651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3d313e4587f4c0ba5d74aa38e750e32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:16.582637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3d313e4587f4c0ba5d74aa38e750e32'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:16.607012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 906/1850] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:16 2022] DELETE /identity/v3/users/a3d313e4587f4c0ba5d74aa38e750e32 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:16.611617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:16.611939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:16.612180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:16.636972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:16.638483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:16.745761 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 945/1851] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:16.760907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:16.763233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:16.847316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d090d2-19a4-44e0-a63a-87e8ca04c9ce admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:16.852415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6e7ec-06b3-4a99-8aac-19435326be9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:16.853062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6e7ec-06b3-4a99-8aac-19435326be9e service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:16.853376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6e7ec-06b3-4a99-8aac-19435326be9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:16.854224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6e7ec-06b3-4a99-8aac-19435326be9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:16.865858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6e7ec-06b3-4a99-8aac-19435326be9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:16.867306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6e7ec-06b3-4a99-8aac-19435326be9e service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:16.880365 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 907/1852] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:17.324933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:17.327209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:17.344824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fcb4f-9585-4ea9-88da-b4036d6fb844 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:17.353272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:17.354728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:17.355091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:17.356224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] PATH_INFO: `/v3/projects/47712cb5b4834f918d3c4437f5bb0feb` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:17.357696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] RBAC: Authorizing `identity:delete_project(project_id=47712cb5b4834f918d3c4437f5bb0feb)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:17.361123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:17.438320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47712cb5b4834f918d3c4437f5bb0feb', 'request_id': 'req-f12c1110-ceea-40a9-bd0d-11fef0a07faa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:17.450870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47712cb5b4834f918d3c4437f5bb0feb', 'request_id': 'req-f12c1110-ceea-40a9-bd0d-11fef0a07faa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:17.464877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47712cb5b4834f918d3c4437f5bb0feb', 'request_id': 'req-f12c1110-ceea-40a9-bd0d-11fef0a07faa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:17.480079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] The token cache is being invalidate because project 47712cb5b4834f918d3c4437f5bb0feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:17.480521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:17.481246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f12c1110-ceea-40a9-bd0d-11fef0a07faa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:17.483047 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 946/1853] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:17 2022] DELETE /identity/v3/projects/47712cb5b4834f918d3c4437f5bb0feb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:17.497608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59cae9-831f-49f2-b085-287591265859 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:17.498023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59cae9-831f-49f2-b085-287591265859 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:17.498415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59cae9-831f-49f2-b085-287591265859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:17.525653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4b59cae9-831f-49f2-b085-287591265859 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:17.527001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4b59cae9-831f-49f2-b085-287591265859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:17.571943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:17.619753 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 908/1854] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:57:17.628091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821081-66f4-417a-92d0-c2e041f6bb08 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:17.628771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821081-66f4-417a-92d0-c2e041f6bb08 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:17.629056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1821081-66f4-417a-92d0-c2e041f6bb08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:17.650023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c1821081-66f4-417a-92d0-c2e041f6bb08 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:17.651245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c1821081-66f4-417a-92d0-c2e041f6bb08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:17.660002 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 947/1855] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:57:17.664206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59cae9-831f-49f2-b085-287591265859 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:17.676887 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59cae9-831f-49f2-b085-287591265859 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:17.681768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:17.682895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:17.682895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:17.683161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:17.683956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:57:17.684457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:17.686148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:17.711058 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 909/1856] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:57:17.711784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821081-66f4-417a-92d0-c2e041f6bb08 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:17.723217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1821081-66f4-417a-92d0-c2e041f6bb08 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:17.729949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:17.730564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:17.730842 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:17.731117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:17.731786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:17.733885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:17.777209 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 948/1857] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:57:17 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:57:17.781656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:17.793376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d0bc5-fe96-46d4-a64f-e9e83c19f076 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:17.897774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:17.898514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:17.898689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:17.898929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:17.899558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:17.902910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:17.931289 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 910/1858] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:57:17 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:17.935108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:17.948371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a451e6-b715-416e-a702-ec7c46afda7d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:17.953484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:17.954057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:17.954337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:17.954615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:17.955344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:17.956896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:17.980237 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 949/1859] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:17.981799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:17.993754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd51bde-bebc-44d7-ae9c-96a51013a89d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:17.999907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d192a8c-2569-4a88-97dc-ae226a00672d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:18.000828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d192a8c-2569-4a88-97dc-ae226a00672d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:18.001334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d192a8c-2569-4a88-97dc-ae226a00672d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:18.001708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d192a8c-2569-4a88-97dc-ae226a00672d admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:18.002925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d192a8c-2569-4a88-97dc-ae226a00672d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:18.004826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d192a8c-2569-4a88-97dc-ae226a00672d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:18.020025 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 911/1860] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:18.021364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:18.035287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e29f3b-e9d1-408c-9439-a85368b9025b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:18.040010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:18.040708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:18.040993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:18.041271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] PATH_INFO: `/v3/projects/522410e2a33a4dc0ac13b1a729a22aa4/users/7cb8aa7495994d2c9168e7734f8a4b6d/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:18.051294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:18.061069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] RBAC: Authorizing `identity:create_grant(project_id=522410e2a33a4dc0ac13b1a729a22aa4, user_id=7cb8aa7495994d2c9168e7734f8a4b6d, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:18.064390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:18.099161 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 950/1861] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:57:18 2022] PUT /identity/v3/projects/522410e2a33a4dc0ac13b1a729a22aa4/users/7cb8aa7495994d2c9168e7734f8a4b6d/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:18.111838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee007e-89f5-49c1-8456-ae224e5195fa None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:18.112260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee007e-89f5-49c1-8456-ae224e5195fa None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:18.112765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ee007e-89f5-49c1-8456-ae224e5195fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:18.161926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-35ee007e-89f5-49c1-8456-ae224e5195fa None None] MFA Rules not processed for user `7cb8aa7495994d2c9168e7734f8a4b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:18.163726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-35ee007e-89f5-49c1-8456-ae224e5195fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:18.236414 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 912/1862] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:18.248253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:18.250726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:18.333029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3151f-0548-48c5-90b0-8be4034fd38d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:18.337246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959fc393-544f-47bf-bc04-2a5f1e5be64a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:18.338030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc393-544f-47bf-bc04-2a5f1e5be64a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:18.338349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc393-544f-47bf-bc04-2a5f1e5be64a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:18.338746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959fc393-544f-47bf-bc04-2a5f1e5be64a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:18.351747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fc393-544f-47bf-bc04-2a5f1e5be64a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:18.353294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959fc393-544f-47bf-bc04-2a5f1e5be64a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:18.364111 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 951/1863] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:18.392358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee007e-89f5-49c1-8456-ae224e5195fa None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:18.448939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:18.511816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ee007e-89f5-49c1-8456-ae224e5195fa None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:18.517390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:18.518031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:18.518308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:18.518603 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] PATH_INFO: `/v3/users/7cb8aa7495994d2c9168e7734f8a4b6d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:18.520509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb8aa7495994d2c9168e7734f8a4b6d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:18.522876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:18.589074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb8aa7495994d2c9168e7734f8a4b6d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:18.590177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb8aa7495994d2c9168e7734f8a4b6d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:18.598732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb8aa7495994d2c9168e7734f8a4b6d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:18.632483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb8aa7495994d2c9168e7734f8a4b6d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:18.633912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb8aa7495994d2c9168e7734f8a4b6d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:18.657620 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 913/1864] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:18 2022] DELETE /identity/v3/users/7cb8aa7495994d2c9168e7734f8a4b6d => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:18.662856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:18.663146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:18.663422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:18.687550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:18.688918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:18.774122 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 952/1865] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:18.784627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:18.786373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:18.882465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f715e4-79fe-4dc0-804f-1ee5c9ac0121 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:18.891285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfca9706-d928-4fc1-8520-b0986b05f1a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:18.892130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca9706-d928-4fc1-8520-b0986b05f1a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:18.892381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca9706-d928-4fc1-8520-b0986b05f1a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:18.892651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfca9706-d928-4fc1-8520-b0986b05f1a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:18.903838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca9706-d928-4fc1-8520-b0986b05f1a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:18.906759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfca9706-d928-4fc1-8520-b0986b05f1a4 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:18.919473 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 914/1866] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:19.274152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:19.276268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:19.290702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3d807b-802a-43ac-b5dd-a43a3a59cf1e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:19.302887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:19.303724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:19.304153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:19.304993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] PATH_INFO: `/v3/projects/522410e2a33a4dc0ac13b1a729a22aa4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:19.306535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=522410e2a33a4dc0ac13b1a729a22aa4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:19.308856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:19.386684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '522410e2a33a4dc0ac13b1a729a22aa4', 'request_id': 'req-e1eaf823-0da3-4744-a510-1a30c93e6d2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:19.396448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '522410e2a33a4dc0ac13b1a729a22aa4', 'request_id': 'req-e1eaf823-0da3-4744-a510-1a30c93e6d2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:19.403245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '522410e2a33a4dc0ac13b1a729a22aa4', 'request_id': 'req-e1eaf823-0da3-4744-a510-1a30c93e6d2c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:19.415663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] The token cache is being invalidate because project 522410e2a33a4dc0ac13b1a729a22aa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:19.416186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:19.417057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:19.418822 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 953/1867] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:19 2022] DELETE /identity/v3/projects/522410e2a33a4dc0ac13b1a729a22aa4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:28.914716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=67,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:29.412358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:36.236979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:36.237544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:36.237544 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:36.257105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:36.258299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:36.311883 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 915/1868] 10.222.0.39 () {58 vars in 967 bytes} [Sun Oct 23 00:57:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:36.324669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:36.326026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:36.393574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaf823-0da3-4744-a510-1a30c93e6d2c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:36.398385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:36.398977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:36.399253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:36.399523 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:36.407286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:36.408898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:36.416668 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 954/1869] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 00:57:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:57:38.399954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:38.401428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:38.469198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da80047c-916e-4f69-a17f-2ada0a3390d4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:38.473899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:38.474550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:38.474830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:38.475105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] PATH_INFO: `/v3/users/08616a72155d4b8595bb7d813b155ad0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:38.476246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08616a72155d4b8595bb7d813b155ad0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:38.478183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:38.527208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08616a72155d4b8595bb7d813b155ad0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.527857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08616a72155d4b8595bb7d813b155ad0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.535541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08616a72155d4b8595bb7d813b155ad0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.547475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08616a72155d4b8595bb7d813b155ad0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.548156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7b72ec0b-bd6d-4491-93e3-00cf8e7c9358 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08616a72155d4b8595bb7d813b155ad0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.564961 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 916/1870] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:38 2022] DELETE /identity/v3/users/08616a72155d4b8595bb7d813b155ad0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:38.743377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:38.755284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e9f74-249a-43f0-ac3c-d5fdb6e4de46 service swift] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:38.815275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:38.816151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:38.816627 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:38.817082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] PATH_INFO: `/v3/projects/07941785917446599d8b991944f7de9d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:38.818211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07941785917446599d8b991944f7de9d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:38.820131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:38.870968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07941785917446599d8b991944f7de9d', 'request_id': 'req-f1b6152f-7325-4059-bde8-cd76fbce58b6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.880112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07941785917446599d8b991944f7de9d', 'request_id': 'req-f1b6152f-7325-4059-bde8-cd76fbce58b6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.886389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07941785917446599d8b991944f7de9d', 'request_id': 'req-f1b6152f-7325-4059-bde8-cd76fbce58b6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.898068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] The token cache is being invalidate because project 07941785917446599d8b991944f7de9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:38.898353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.898996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f1b6152f-7325-4059-bde8-cd76fbce58b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:38.900515 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 955/1871] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:38 2022] DELETE /identity/v3/projects/07941785917446599d8b991944f7de9d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:38.905621 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db80ba-a65f-41f7-94ef-3ea62c8d9cdf None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:38.905906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db80ba-a65f-41f7-94ef-3ea62c8d9cdf None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:38.906177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db80ba-a65f-41f7-94ef-3ea62c8d9cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:38.928092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b9db80ba-a65f-41f7-94ef-3ea62c8d9cdf None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:38.929186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b9db80ba-a65f-41f7-94ef-3ea62c8d9cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:38.955622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:38.999187 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 917/1872] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:39.007022 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf89dd-c53e-4b05-b351-d3c28eb46539 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.007449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf89dd-c53e-4b05-b351-d3c28eb46539 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.007727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccf89dd-c53e-4b05-b351-d3c28eb46539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.027991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5ccf89dd-c53e-4b05-b351-d3c28eb46539 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:39.029226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5ccf89dd-c53e-4b05-b351-d3c28eb46539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:39.037559 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 956/1873] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:39.041700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db80ba-a65f-41f7-94ef-3ea62c8d9cdf None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:39.051977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db80ba-a65f-41f7-94ef-3ea62c8d9cdf None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:39.056450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c507d-2873-4136-8103-96080163815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:39.057040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c507d-2873-4136-8103-96080163815d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.057318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c507d-2873-4136-8103-96080163815d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.057583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c507d-2873-4136-8103-96080163815d admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.058218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c507d-2873-4136-8103-96080163815d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:57:39.058543 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c507d-2873-4136-8103-96080163815d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:39.060103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827c507d-2873-4136-8103-96080163815d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:39.072096 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 918/1874] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:57:39.076772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf89dd-c53e-4b05-b351-d3c28eb46539 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:39.085863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccf89dd-c53e-4b05-b351-d3c28eb46539 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:39.090480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:39.091111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.091392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.091664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.092846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:39.094732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:39.105749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:39.122908 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 957/1875] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:57:39 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:39.127100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c507d-2873-4136-8103-96080163815d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:39.137020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c507d-2873-4136-8103-96080163815d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:39.192718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:39.193349 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.193633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.193905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.194529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:39.196459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:39.218175 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 919/1876] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:57:39 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:39.222297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:39.232448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabd0852-d658-4e70-985a-bdc9963f173a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:39.237065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:39.237663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.237946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.238219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.238857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:39.240466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:39.250207 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 958/1877] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:39.254127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:39.264364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf5af37-f569-41ae-81a0-05f2e703ebd1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:39.268899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd1a42c-2f45-4a7d-b483-603e59f5c90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:39.269491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1a42c-2f45-4a7d-b483-603e59f5c90d admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.269770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1a42c-2f45-4a7d-b483-603e59f5c90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.270040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd1a42c-2f45-4a7d-b483-603e59f5c90d admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.270676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1a42c-2f45-4a7d-b483-603e59f5c90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:39.272225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd1a42c-2f45-4a7d-b483-603e59f5c90d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:39.283298 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 920/1878] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:57:39.287939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:39.297982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239b927-1bab-464f-a2de-6e2931a4bad2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:39.302959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:39.303772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.304054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.304348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] PATH_INFO: `/v3/projects/2cac3654e3414c61a1c14fed10f23629/users/aff40c9715b34e64846cab3c2bb40a0d/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.318718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cac3654e3414c61a1c14fed10f23629, user_id=aff40c9715b34e64846cab3c2bb40a0d, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:39.321452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:39.334449 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 959/1879] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:57:39 2022] PUT /identity/v3/projects/2cac3654e3414c61a1c14fed10f23629/users/aff40c9715b34e64846cab3c2bb40a0d/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:39.401361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdce9d-b0c2-477a-803b-ebb7a02e4b48 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.401904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdce9d-b0c2-477a-803b-ebb7a02e4b48 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.401904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdce9d-b0c2-477a-803b-ebb7a02e4b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.429334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6efdce9d-b0c2-477a-803b-ebb7a02e4b48 None None] MFA Rules not processed for user `aff40c9715b34e64846cab3c2bb40a0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:39.430567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6efdce9d-b0c2-477a-803b-ebb7a02e4b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:39.493623 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 921/1880] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:39.505863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:39.507541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:39.584151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcef3e14-92c0-48ac-a9a3-90a2d378009d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:39.588368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:39.588973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:39.589249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:39.589521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:39.598417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:39.599864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:39.609174 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 960/1881] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:46.161389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3839d-1abc-4524-9ed0-a09f90eb4e11 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:46.161958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3839d-1abc-4524-9ed0-a09f90eb4e11 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:46.161958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3839d-1abc-4524-9ed0-a09f90eb4e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:46.181040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4df3839d-1abc-4524-9ed0-a09f90eb4e11 None None] MFA Rules not processed for user `28563be0db564f8ebcf765a78970ce8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:46.182211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4df3839d-1abc-4524-9ed0-a09f90eb4e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:46.196191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:46.242485 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 922/1882] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:46 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:46.953470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:46.955161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:47.028173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26731bb-f68e-4798-85fd-d6bfc2f161d7 service glance] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:47.033241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:47.033863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.034176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.034443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] PATH_INFO: `/v3/users/4bf78b2498104b05b8135e82a93b4a7b` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.035932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bf78b2498104b05b8135e82a93b4a7b)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:47.038310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:47.088750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:47.095332 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bf78b2498104b05b8135e82a93b4a7b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.105111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bf78b2498104b05b8135e82a93b4a7b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.115025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bf78b2498104b05b8135e82a93b4a7b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.129070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bf78b2498104b05b8135e82a93b4a7b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.129771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bf78b2498104b05b8135e82a93b4a7b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.135955 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 961/1883] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:57:46 2022] DELETE /identity/v3/users/4bf78b2498104b05b8135e82a93b4a7b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:47.143498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.143498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.143498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.167769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:47.168971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:47.235709 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 923/1884] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:57:47.245601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:47.247098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:47.308005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7621d-6a13-455a-b070-0cc2d3ba0ea2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:47.313552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ea724-a716-4386-9fd6-1089edfac07d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:47.314294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea724-a716-4386-9fd6-1089edfac07d service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.314656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea724-a716-4386-9fd6-1089edfac07d service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.314926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ea724-a716-4386-9fd6-1089edfac07d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.324967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea724-a716-4386-9fd6-1089edfac07d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:47.326535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ea724-a716-4386-9fd6-1089edfac07d service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:47.337991 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 962/1885] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:47.614197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:47.616003 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:47.638604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f84bf1-1523-4b1d-a4ca-dceccf291b63 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:47.646301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:47.646949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.647225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.647494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] PATH_INFO: `/v3/projects/89b93cbb32fd414c8668488f5dcd0b21` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.648642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89b93cbb32fd414c8668488f5dcd0b21)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:47.650555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:47.706215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89b93cbb32fd414c8668488f5dcd0b21', 'request_id': 'req-1e4def08-29ad-4fbc-8600-ec89ad5a3456'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.713743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89b93cbb32fd414c8668488f5dcd0b21', 'request_id': 'req-1e4def08-29ad-4fbc-8600-ec89ad5a3456'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.719999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89b93cbb32fd414c8668488f5dcd0b21', 'request_id': 'req-1e4def08-29ad-4fbc-8600-ec89ad5a3456'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.733054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] The token cache is being invalidate because project 89b93cbb32fd414c8668488f5dcd0b21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:57:47.733420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.734177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e4def08-29ad-4fbc-8600-ec89ad5a3456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:57:47.736233 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 924/1886] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:57:47 2022] DELETE /identity/v3/projects/89b93cbb32fd414c8668488f5dcd0b21 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:47.741125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc8fd9-15c1-40ff-91e6-45e92bd6a229 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.741406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc8fd9-15c1-40ff-91e6-45e92bd6a229 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.741674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc8fd9-15c1-40ff-91e6-45e92bd6a229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.761883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d6cc8fd9-15c1-40ff-91e6-45e92bd6a229 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:47.763130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d6cc8fd9-15c1-40ff-91e6-45e92bd6a229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:47.824709 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 963/1887] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:47.832952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713bc1-dcd9-4145-8dff-771d521a6e66 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.833235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713bc1-dcd9-4145-8dff-771d521a6e66 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.833506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98713bc1-dcd9-4145-8dff-771d521a6e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.854444 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-98713bc1-dcd9-4145-8dff-771d521a6e66 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:47.855632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-98713bc1-dcd9-4145-8dff-771d521a6e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:47.864085 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 925/1888] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:47.868478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc8fd9-15c1-40ff-91e6-45e92bd6a229 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:47.879581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc8fd9-15c1-40ff-91e6-45e92bd6a229 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:47.884416 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:47.885040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.885316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.885584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.886228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:57:47.886527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:47.888148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:47.901018 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 964/1889] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:57:47.905169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713bc1-dcd9-4145-8dff-771d521a6e66 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:47.914795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98713bc1-dcd9-4145-8dff-771d521a6e66 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:47.919774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:47.920525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:47.920852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:47.921202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:47.922556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:47.924488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:47.956467 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 926/1890] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:57:47 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 00:57:47.960638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:47.970976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c46d3f-728e-4e78-8ea5-7d90ac7d1f4f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:48.028837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:48.038344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:48.039081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:48.039365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:48.039635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:48.040325 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:48.042305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:48.069064 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 965/1891] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:57:47 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:57:48.070229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:48.080745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08765225-3b09-4380-8991-73073beed4ac admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:48.086306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:48.086962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:48.087186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:48.087550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:48.088120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:48.089736 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:48.103389 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 927/1892] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:57:48.104598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:48.115519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e06f167-8bb9-45be-9fe8-67be5b75cb96 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:48.123128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07cd2c4-5e02-4748-9907-46e44e0b7056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:48.123854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cd2c4-5e02-4748-9907-46e44e0b7056 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:48.124190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cd2c4-5e02-4748-9907-46e44e0b7056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:48.124614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07cd2c4-5e02-4748-9907-46e44e0b7056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:48.125889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cd2c4-5e02-4748-9907-46e44e0b7056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:48.127828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07cd2c4-5e02-4748-9907-46e44e0b7056 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:48.140104 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 966/1893] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:57:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 00:57:48.145406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:48.156041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f03ba5-1ea4-4c34-a4ba-f5b07771c560 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:48.161152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:48.161954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:48.162328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:48.162692 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] PATH_INFO: `/v3/projects/bd4636922d3f4936b68c2d38f53ab4d4/users/4359f42a787a44fd97579a46a246f27a/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:48.169362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:57:48.186712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd4636922d3f4936b68c2d38f53ab4d4, user_id=4359f42a787a44fd97579a46a246f27a, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:48.191095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:48.212651 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 928/1894] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:57:48 2022] PUT /identity/v3/projects/bd4636922d3f4936b68c2d38f53ab4d4/users/4359f42a787a44fd97579a46a246f27a/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:57:48.214661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce483e0-e789-41de-964b-45f4f79d4f7e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:48.215032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce483e0-e789-41de-964b-45f4f79d4f7e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:48.216000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce483e0-e789-41de-964b-45f4f79d4f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:48.236996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6ce483e0-e789-41de-964b-45f4f79d4f7e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:48.239446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6ce483e0-e789-41de-964b-45f4f79d4f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:48.306059 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 967/1895] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:57:48.316467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:48.318133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:48.378821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aed49b-a7fb-40c8-bda6-bb478617069a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:48.382796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:48.383415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:48.383694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:48.383964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:48.394114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:48.395540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:48.406930 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 929/1896] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:49.131679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6027b8-259b-4450-a11d-9c1cab176d4f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:49.132155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6027b8-259b-4450-a11d-9c1cab176d4f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:49.132528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6027b8-259b-4450-a11d-9c1cab176d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:49.180147 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6f6027b8-259b-4450-a11d-9c1cab176d4f None None] MFA Rules not processed for user `4359f42a787a44fd97579a46a246f27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:49.181478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6f6027b8-259b-4450-a11d-9c1cab176d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:49.260093 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 968/1897] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:49.272552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:57:49.274072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:49.353081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7c16b8-92b3-4354-81b6-470fb76fc167 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:57:49.360321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:57:49.361191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:49.361681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:49.362143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:49.374512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:57:49.376454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:57:49.388248 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 930/1898] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:57:49 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:57:50.808063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe03e4a-907d-4176-a512-fa7c0d474c0b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:57:50.808640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe03e4a-907d-4176-a512-fa7c0d474c0b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:57:50.809050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe03e4a-907d-4176-a512-fa7c0d474c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:57:50.828672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-abe03e4a-907d-4176-a512-fa7c0d474c0b None None] MFA Rules not processed for user `4359f42a787a44fd97579a46a246f27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:57:50.829874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-abe03e4a-907d-4176-a512-fa7c0d474c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:57:50.837899 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 969/1899] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:57:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:57:59.383930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=58,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:00.823710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:08.673582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:08.675092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:08.688813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7296f-e9aa-45dd-ad58-a1e9a0e011f0 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:08.693972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:08.694362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:08.694635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:08.695030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] PATH_INFO: `/v3/users/aff40c9715b34e64846cab3c2bb40a0d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:08.696183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff40c9715b34e64846cab3c2bb40a0d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:08.697968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:08.745523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff40c9715b34e64846cab3c2bb40a0d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:08.746179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff40c9715b34e64846cab3c2bb40a0d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:08.754611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff40c9715b34e64846cab3c2bb40a0d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:08.766984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff40c9715b34e64846cab3c2bb40a0d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:08.768385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff40c9715b34e64846cab3c2bb40a0d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:08.781047 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 931/1900] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 00:58:08 2022] DELETE /identity/v3/users/aff40c9715b34e64846cab3c2bb40a0d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:58:08.785517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:08.785990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:08.786075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:08.805968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:08.807847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:08.876239 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 970/1901] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:08.886725 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:08.888825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:08.954416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530afe1c-d350-4183-aa38-834ffb03efc0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:08.958933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c6a3ad-422b-43ae-a7a6-a803e5b6488a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:08.959565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6a3ad-422b-43ae-a7a6-a803e5b6488a service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:08.959854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6a3ad-422b-43ae-a7a6-a803e5b6488a service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:08.960122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c6a3ad-422b-43ae-a7a6-a803e5b6488a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:08.969343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6a3ad-422b-43ae-a7a6-a803e5b6488a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:08.970905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c6a3ad-422b-43ae-a7a6-a803e5b6488a service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:08.980123 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 932/1902] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:09.261440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.263051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:09.278751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e5108-5cee-4a86-97f5-b0083646194e None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.284199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.285055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.285770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.285770 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] PATH_INFO: `/v3/projects/2cac3654e3414c61a1c14fed10f23629` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.287155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cac3654e3414c61a1c14fed10f23629)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.290790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.340233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cac3654e3414c61a1c14fed10f23629', 'request_id': 'req-f56ea503-ea35-409d-8f48-c02aba2d499a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:09.347515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cac3654e3414c61a1c14fed10f23629', 'request_id': 'req-f56ea503-ea35-409d-8f48-c02aba2d499a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:09.353737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cac3654e3414c61a1c14fed10f23629', 'request_id': 'req-f56ea503-ea35-409d-8f48-c02aba2d499a'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:09.364705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] The token cache is being invalidate because project 2cac3654e3414c61a1c14fed10f23629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 00:58:09.364990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:09.365610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-f56ea503-ea35-409d-8f48-c02aba2d499a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 00:58:09.367338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 971/1903] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 00:58:09 2022] DELETE /identity/v3/projects/2cac3654e3414c61a1c14fed10f23629 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:58:09.372110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fed70-10a5-4185-b03d-0ad594fddd0e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.372426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fed70-10a5-4185-b03d-0ad594fddd0e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.372699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1fed70-10a5-4185-b03d-0ad594fddd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.392797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1b1fed70-10a5-4185-b03d-0ad594fddd0e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:09.393989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b1fed70-10a5-4185-b03d-0ad594fddd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:09.452606 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 933/1904] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:09.460511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984208e7-d94f-4bad-a1d9-85e3b3a7f2c3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.460798 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984208e7-d94f-4bad-a1d9-85e3b3a7f2c3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.461103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984208e7-d94f-4bad-a1d9-85e3b3a7f2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.481264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-984208e7-d94f-4bad-a1d9-85e3b3a7f2c3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:09.482451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-984208e7-d94f-4bad-a1d9-85e3b3a7f2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:09.490897 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 972/1905] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:09.495478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fed70-10a5-4185-b03d-0ad594fddd0e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.505752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1fed70-10a5-4185-b03d-0ad594fddd0e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.510658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.511266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.511541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.511808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.512488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 00:58:09.512788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.514425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.526318 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 934/1906] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 00:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 00:58:09.530781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984208e7-d94f-4bad-a1d9-85e3b3a7f2c3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.540554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984208e7-d94f-4bad-a1d9-85e3b3a7f2c3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.545448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.546048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.546329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.546600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.547223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.549757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.577838 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 973/1907] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:58:09 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:58:09.582134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.587143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:09.594105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401bb096-1c9f-4ee3-9e5e-3bf495e987b1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.654261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.654862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.655142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.655415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.656058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.658606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.680911 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 935/1908] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:58:09 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:58:09.685073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.695293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e4d9a-c64d-4b93-9932-e072e36c3f12 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.700097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.700737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.701014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.701303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.701923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.703549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.713728 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 974/1909] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:58:09.717794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.729379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ea227-b801-4c24-8a16-2298b5f2bccf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.733926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdf56f-83b9-49b0-9670-895e790ae5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.734527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf56f-83b9-49b0-9670-895e790ae5ec admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.734801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf56f-83b9-49b0-9670-895e790ae5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.735069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdf56f-83b9-49b0-9670-895e790ae5ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.735701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdf56f-83b9-49b0-9670-895e790ae5ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.737859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdf56f-83b9-49b0-9670-895e790ae5ec admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.748809 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 936/1910] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:58:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:58:09.753369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.763115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c698a80-7a1a-4917-96f9-c4e577d208a2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.769102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.769776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.770053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.770321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] PATH_INFO: `/v3/projects/ac18f8515a94410891c7675051d1ac59/users/51c96b11cfe540debe7e32c1d5b915a7/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.785868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac18f8515a94410891c7675051d1ac59, user_id=51c96b11cfe540debe7e32c1d5b915a7, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.788678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.802477 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 975/1911] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:58:09 2022] PUT /identity/v3/projects/ac18f8515a94410891c7675051d1ac59/users/51c96b11cfe540debe7e32c1d5b915a7/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:58:09.807858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd091e8-6e01-4dd8-9a57-2a67694b6bae None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.808153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd091e8-6e01-4dd8-9a57-2a67694b6bae None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.808468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd091e8-6e01-4dd8-9a57-2a67694b6bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.827723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3cd091e8-6e01-4dd8-9a57-2a67694b6bae None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:09.828944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3cd091e8-6e01-4dd8-9a57-2a67694b6bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:09.888313 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 937/1912] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:58:09.898535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:09.900227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:09.921914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:09.959016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6decd-09d0-49e3-8d88-6f08e3301361 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:09.962536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:09.963146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:09.963477 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:09.963753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:09.973030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:09.974444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:09.983691 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 976/1913] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:14.239486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df2e0e-ecc4-480a-a266-0c0e859b86ea None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:14.240336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df2e0e-ecc4-480a-a266-0c0e859b86ea None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:14.240690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df2e0e-ecc4-480a-a266-0c0e859b86ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:14.272337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a5df2e0e-ecc4-480a-a266-0c0e859b86ea None None] MFA Rules not processed for user `51c96b11cfe540debe7e32c1d5b915a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:14.273639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a5df2e0e-ecc4-480a-a266-0c0e859b86ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:14.395860 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 938/1914] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:14.402011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:14.411932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a1d2d-8af5-42e1-ad1e-a9082c5d1502 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:14.417561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:14.421599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:14.421892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:14.422171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:14.422837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:14.425420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:14.463893 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 977/1915] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 00:58:14 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:58:14.468289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df2e0e-ecc4-480a-a266-0c0e859b86ea None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:14.481682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df2e0e-ecc4-480a-a266-0c0e859b86ea None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:14.503342 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:14.559052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:14.560101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:14.560334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:14.560682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:14.561358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:14.563361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:14.587014 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 939/1916] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 00:58:14 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 00:58:14.591142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:14.602437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecdff43-33fb-438b-b100-92f8a7b54705 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:14.607683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:14.608869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:14.608991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:14.609201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:14.609851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:14.612236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:14.626654 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 978/1917] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:58:14.629689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:14.641624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37e9d3-eaf8-4113-9f4e-2d904154f12f admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:14.646604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:14.647281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:14.647560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:14.647859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] PATH_INFO: `/v3/projects/99d6804d5c474f00ab3abbceab6f4165/users/fef5184f076e43a8bbf058b99467c2f5/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:14.668130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d6804d5c474f00ab3abbceab6f4165, user_id=fef5184f076e43a8bbf058b99467c2f5, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:14.670949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:14.684474 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 940/1918] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:58:14 2022] PUT /identity/v3/projects/99d6804d5c474f00ab3abbceab6f4165/users/fef5184f076e43a8bbf058b99467c2f5/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 00:58:14.688622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:14.699070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e899d227-4eab-401a-a33f-8e520aa18dde admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:14.758066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c922d-55dd-4ddc-8dfe-cd4e8b07365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:14.758826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c922d-55dd-4ddc-8dfe-cd4e8b07365a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:14.759107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c922d-55dd-4ddc-8dfe-cd4e8b07365a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:14.759375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c922d-55dd-4ddc-8dfe-cd4e8b07365a admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:14.760016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c922d-55dd-4ddc-8dfe-cd4e8b07365a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:14.761635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c922d-55dd-4ddc-8dfe-cd4e8b07365a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:14.772894 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 979/1919] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 00:58:14 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 00:58:14.776675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:14.788061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628b1f7-3ae6-4d56-942e-c6d4a5dae7c1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:14.793090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91266f72-557b-4642-a473-93791b62099c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:14.793997 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266f72-557b-4642-a473-93791b62099c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:14.794274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266f72-557b-4642-a473-93791b62099c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:14.794547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91266f72-557b-4642-a473-93791b62099c admin admin] PATH_INFO: `/v3/projects/99d6804d5c474f00ab3abbceab6f4165/users/fef5184f076e43a8bbf058b99467c2f5/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:14.796667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91266f72-557b-4642-a473-93791b62099c admin admin] RBAC: Authorizing `identity:create_grant(project_id=99d6804d5c474f00ab3abbceab6f4165, user_id=fef5184f076e43a8bbf058b99467c2f5, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:14.799857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91266f72-557b-4642-a473-93791b62099c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:14.811328 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 941/1920] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 00:58:14 2022] PUT /identity/v3/projects/99d6804d5c474f00ab3abbceab6f4165/users/fef5184f076e43a8bbf058b99467c2f5/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 00:58:19.866102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9be6c-6e2f-44e3-9fe5-419f35c851f4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:19.866598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9be6c-6e2f-44e3-9fe5-419f35c851f4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:19.866676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9be6c-6e2f-44e3-9fe5-419f35c851f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:19.897232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-57f9be6c-6e2f-44e3-9fe5-419f35c851f4 None None] MFA Rules not processed for user `fef5184f076e43a8bbf058b99467c2f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:19.898582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-57f9be6c-6e2f-44e3-9fe5-419f35c851f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:19.969684 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 980/1921] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:19.983993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91266f72-557b-4642-a473-93791b62099c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:19.985983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-91266f72-557b-4642-a473-93791b62099c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:20.119311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91266f72-557b-4642-a473-93791b62099c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:20.125154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:20.125855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:20.126196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:20.127068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:20.223917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:20.225697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:20.231031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:20.240591 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 942/1922] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:21.404826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655fc1d-6f97-4f89-a644-5b70a5b8aed9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:21.405316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655fc1d-6f97-4f89-a644-5b70a5b8aed9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:21.405396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655fc1d-6f97-4f89-a644-5b70a5b8aed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:21.426854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4655fc1d-6f97-4f89-a644-5b70a5b8aed9 None None] MFA Rules not processed for user `51c96b11cfe540debe7e32c1d5b915a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:21.428152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4655fc1d-6f97-4f89-a644-5b70a5b8aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:21.435479 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 981/1923] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:23.238036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:23.242432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:23.367004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ed5777-eb4b-42de-b54b-775d11a26dd4 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:23.372067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:23.372863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:23.373249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:23.373620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:23.375455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:23.389562 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:23.390731 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:23.411654 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 943/1924] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:31.421701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:31.721087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eb427-5d17-457b-be05-4ef4f7c6a005 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:31.721654 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eb427-5d17-457b-be05-4ef4f7c6a005 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:31.722003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9eb427-5d17-457b-be05-4ef4f7c6a005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:31.745107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1b9eb427-5d17-457b-be05-4ef4f7c6a005 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:31.746823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1b9eb427-5d17-457b-be05-4ef4f7c6a005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:31.804536 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 982/1925] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 00:58:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:31.820643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:31.832584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102661b0-15e4-4d20-a71f-8190d0464d5f service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:31.837813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:31.838406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:31.838682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:31.838950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:31.847331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:31.848913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:31.858094 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 944/1926] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:36.941825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bf01-099c-4fb4-aac9-c82a546f4383 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:36.942344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bf01-099c-4fb4-aac9-c82a546f4383 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:36.942448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b184bf01-099c-4fb4-aac9-c82a546f4383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:36.961757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b184bf01-099c-4fb4-aac9-c82a546f4383 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:36.962926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b184bf01-099c-4fb4-aac9-c82a546f4383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:36.971353 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 983/1927] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 00:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:58:36.990160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:36.991539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:37.006709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6a201-4351-4d09-bf3c-180fe2f51cc5 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:37.012101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:37.012722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:37.013002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:37.013268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:37.021268 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:37.022822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:37.032074 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 945/1928] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:40.412967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e14a-8dc3-42b5-9a5f-0f09f178bdf7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:40.413479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e14a-8dc3-42b5-9a5f-0f09f178bdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:40.413479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1e14a-8dc3-42b5-9a5f-0f09f178bdf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:40.435555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-02a1e14a-8dc3-42b5-9a5f-0f09f178bdf7 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:40.436836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-02a1e14a-8dc3-42b5-9a5f-0f09f178bdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:40.494564 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 984/1929] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:58:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:40.540270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:40.541730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:40.612738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3b1fe-8726-41dc-81ac-b651f725bb41 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:40.617712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3a58ef-3fc7-4ffc-adb8-32000823fefc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:40.618316 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a58ef-3fc7-4ffc-adb8-32000823fefc service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:40.618598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a58ef-3fc7-4ffc-adb8-32000823fefc service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:40.618873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a58ef-3fc7-4ffc-adb8-32000823fefc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:40.627323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a58ef-3fc7-4ffc-adb8-32000823fefc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:40.629179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3a58ef-3fc7-4ffc-adb8-32000823fefc service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:40.637034 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 946/1930] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 00:58:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:58:42.291594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e516535-69b0-461c-a5e3-347e5198f76c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:42.296582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e516535-69b0-461c-a5e3-347e5198f76c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:42.296936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e516535-69b0-461c-a5e3-347e5198f76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:42.330686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75b83a-a263-43c1-b16e-020426af0156 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:42.331119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75b83a-a263-43c1-b16e-020426af0156 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:42.331448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75b83a-a263-43c1-b16e-020426af0156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:42.359340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2e516535-69b0-461c-a5e3-347e5198f76c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:42.361156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2e516535-69b0-461c-a5e3-347e5198f76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:42.371172 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 985/1931] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:42.389191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-af75b83a-a263-43c1-b16e-020426af0156 None None] MFA Rules not processed for user `959991a17c69459dbd665c2ae20095ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:42.394530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-af75b83a-a263-43c1-b16e-020426af0156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:42.467156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:58:42.478967 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 947/1932] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:58:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:58:42.621127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e516535-69b0-461c-a5e3-347e5198f76c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:42.634902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e516535-69b0-461c-a5e3-347e5198f76c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:42.643860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:42.644707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:42.644987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:42.645258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:42.656610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:42.658529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:42.670963 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 986/1933] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:47.544656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736b5e7-0b61-4d5e-87e3-1ba2ffd991f7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:47.545684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736b5e7-0b61-4d5e-87e3-1ba2ffd991f7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:47.546173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736b5e7-0b61-4d5e-87e3-1ba2ffd991f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:47.579823 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e736b5e7-0b61-4d5e-87e3-1ba2ffd991f7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:47.581224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e736b5e7-0b61-4d5e-87e3-1ba2ffd991f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:47.592226 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 948/1934] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 00:58:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:47.706252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:47.724258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca24a4-d641-4d5b-a79f-208d6e73c0ca service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:47.730059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:47.730780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:47.731406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:47.731790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:47.742987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:47.747051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:47.759592 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 987/1935] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:58:55.944714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208130c-6ea8-4475-8080-7e0a5002bae9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:55.945320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208130c-6ea8-4475-8080-7e0a5002bae9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:55.946469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6208130c-6ea8-4475-8080-7e0a5002bae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:55.965451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6208130c-6ea8-4475-8080-7e0a5002bae9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:55.966639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6208130c-6ea8-4475-8080-7e0a5002bae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:55.975045 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 949/1936] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 00:58:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:55.990370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:56.001402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dbdc59-699b-49bf-aaba-d6fb01f8a8c0 service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:56.007514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e24bed-db0f-4563-a56c-47b25a42d5c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:56.008116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e24bed-db0f-4563-a56c-47b25a42d5c9 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:56.008430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e24bed-db0f-4563-a56c-47b25a42d5c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:56.008860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e24bed-db0f-4563-a56c-47b25a42d5c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:56.017679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e24bed-db0f-4563-a56c-47b25a42d5c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:56.019328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e24bed-db0f-4563-a56c-47b25a42d5c9 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:56.028878 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 988/1937] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:58:59.711592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c865-1b1d-486a-aa51-f0d56e1589eb None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:59.712156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c865-1b1d-486a-aa51-f0d56e1589eb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:59.712256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224c865-1b1d-486a-aa51-f0d56e1589eb None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:59.715572 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 950/1938] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:58:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:58:59.721773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17363a-8a3b-4d8b-bc65-2d1f10a0db03 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:59.722238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17363a-8a3b-4d8b-bc65-2d1f10a0db03 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:59.722409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17363a-8a3b-4d8b-bc65-2d1f10a0db03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:59.742034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bb17363a-8a3b-4d8b-bc65-2d1f10a0db03 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:59.743229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bb17363a-8a3b-4d8b-bc65-2d1f10a0db03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:59.753803 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 989/1939] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:58:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:58:59.766156 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c865-1b1d-486a-aa51-f0d56e1589eb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:58:59.768416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2224c865-1b1d-486a-aa51-f0d56e1589eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:58:59.828977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224c865-1b1d-486a-aa51-f0d56e1589eb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:58:59.833790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a66f77-581b-4385-b114-d9efbafe7b60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:58:59.834498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66f77-581b-4385-b114-d9efbafe7b60 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:59.834849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66f77-581b-4385-b114-d9efbafe7b60 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:59.835189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a66f77-581b-4385-b114-d9efbafe7b60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:59.846565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a66f77-581b-4385-b114-d9efbafe7b60 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:58:59.847974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a66f77-581b-4385-b114-d9efbafe7b60 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:58:59.858580 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 951/1940] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:58:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 00:58:59.928245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be87f97-f1a5-42ef-9dea-42fb709df636 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:59.928591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be87f97-f1a5-42ef-9dea-42fb709df636 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:59.928869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be87f97-f1a5-42ef-9dea-42fb709df636 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:59.930228 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 990/1941] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:58:59.934681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9974d3-7609-4a6b-aecc-75af4b2dcc7e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:58:59.934967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9974d3-7609-4a6b-aecc-75af4b2dcc7e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:58:59.935237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9974d3-7609-4a6b-aecc-75af4b2dcc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:58:59.958680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1c9974d3-7609-4a6b-aecc-75af4b2dcc7e None None] MFA Rules not processed for user `bb5f82d3cfc14eacbe4cc40bcd0dfb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:58:59.959933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1c9974d3-7609-4a6b-aecc-75af4b2dcc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:59:00.010221 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 952/1942] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:58:59 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:59:00.022708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be87f97-f1a5-42ef-9dea-42fb709df636 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:59:00.035668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be87f97-f1a5-42ef-9dea-42fb709df636 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:59:00.039279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd615625-b5cf-49be-a78b-08e98fe11de4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:59:00.039876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd615625-b5cf-49be-a78b-08e98fe11de4 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:00.040157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd615625-b5cf-49be-a78b-08e98fe11de4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:00.040586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd615625-b5cf-49be-a78b-08e98fe11de4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:00.047527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd615625-b5cf-49be-a78b-08e98fe11de4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:59:00.049003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd615625-b5cf-49be-a78b-08e98fe11de4 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:59:00.056545 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 991/1943] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:59:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:59:00.137696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e61c0-caee-4434-b3bf-60d3c0d30ea6 None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:00.138083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e61c0-caee-4434-b3bf-60d3c0d30ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:00.138359 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312e61c0-caee-4434-b3bf-60d3c0d30ea6 None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:00.139826 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 953/1944] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 00:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 00:59:00.145458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760fa5-b5a5-47f2-8d13-b069fa2b3f97 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:00.145754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760fa5-b5a5-47f2-8d13-b069fa2b3f97 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:00.146060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91760fa5-b5a5-47f2-8d13-b069fa2b3f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:00.166061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-91760fa5-b5a5-47f2-8d13-b069fa2b3f97 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:59:00.167272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-91760fa5-b5a5-47f2-8d13-b069fa2b3f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:59:00.176489 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 992/1945] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 00:59:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:59:00.186834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e61c0-caee-4434-b3bf-60d3c0d30ea6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:59:00.188545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-312e61c0-caee-4434-b3bf-60d3c0d30ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:59:00.201502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312e61c0-caee-4434-b3bf-60d3c0d30ea6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:59:00.207036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:59:00.207652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:00.207934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:00.208207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:00.218880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:59:00.220942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:59:00.232127 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 954/1946] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:59:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:59:02.205437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a055f-9c26-459d-a5ea-b31218c6bbdd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:02.205951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a055f-9c26-459d-a5ea-b31218c6bbdd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:02.206077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a055f-9c26-459d-a5ea-b31218c6bbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:02.213701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:59:02.226579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-443a055f-9c26-459d-a5ea-b31218c6bbdd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:59:02.227723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-443a055f-9c26-459d-a5ea-b31218c6bbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:59:02.236180 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 993/1947] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 00:59:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:59:02.250377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:59:02.262145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4fc51-8421-4192-baa1-57135a5f5923 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:59:02.268163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:59:02.268824 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:02.269100 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:02.269366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:02.277759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:59:02.279318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:59:02.300621 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 955/1948] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:59:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:59:09.896502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14643437-3352-4049-96e8-c9f9eed6d703 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:09.896966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14643437-3352-4049-96e8-c9f9eed6d703 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:09.897335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14643437-3352-4049-96e8-c9f9eed6d703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:09.916908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-14643437-3352-4049-96e8-c9f9eed6d703 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:59:09.918087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-14643437-3352-4049-96e8-c9f9eed6d703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:59:09.925250 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 994/1949] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 00:59:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 00:59:09.938287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:59:09.949389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa85cf73-1931-4ef4-908d-d0c39a7e4b18 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:59:09.955369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605c2af9-225f-4341-aad5-a004e5e01855 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:59:09.955822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2af9-225f-4341-aad5-a004e5e01855 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:09.956101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2af9-225f-4341-aad5-a004e5e01855 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:09.956417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605c2af9-225f-4341-aad5-a004e5e01855 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:09.964495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c2af9-225f-4341-aad5-a004e5e01855 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:59:09.966046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605c2af9-225f-4341-aad5-a004e5e01855 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:59:09.974552 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 956/1950] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 00:59:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:59:11.860568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14643437-3352-4049-96e8-c9f9eed6d703 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:59:11.871350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14643437-3352-4049-96e8-c9f9eed6d703 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:59:11.875300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:59:11.876127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:11.876445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:11.876717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:11.895713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:59:11.897248 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:59:11.907586 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 995/1951] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 00:59:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 00:59:19.971339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=74 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:59:21.902646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=11 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:59:39.612773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af8fb2-be54-45ac-9a9e-679d672af453 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:39.613330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af8fb2-be54-45ac-9a9e-679d672af453 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:39.613498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af8fb2-be54-45ac-9a9e-679d672af453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:39.635470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-68af8fb2-be54-45ac-9a9e-679d672af453 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:59:39.636761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-68af8fb2-be54-45ac-9a9e-679d672af453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:59:39.644693 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 957/1952] 10.222.0.39 () {58 vars in 967 bytes} [Sun Oct 23 00:59:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:59:39.657757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 00:59:39.669453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d744fe3-a1fb-488e-af2f-edca0211a176 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 00:59:39.674315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44e701-6f51-46be-91b0-ac4034f1b11a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 00:59:39.674918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44e701-6f51-46be-91b0-ac4034f1b11a service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:39.675194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44e701-6f51-46be-91b0-ac4034f1b11a service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:39.675461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44e701-6f51-46be-91b0-ac4034f1b11a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:39.683967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44e701-6f51-46be-91b0-ac4034f1b11a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 00:59:39.685574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44e701-6f51-46be-91b0-ac4034f1b11a service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 00:59:39.693622 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 996/1953] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 00:59:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 00:59:45.163355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d6987-0707-4b01-ac20-ecebbd32b53c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 00:59:45.163850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d6987-0707-4b01-ac20-ecebbd32b53c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 00:59:45.163953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d6987-0707-4b01-ac20-ecebbd32b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 00:59:45.182971 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ff4d6987-0707-4b01-ac20-ecebbd32b53c None None] MFA Rules not processed for user `4359f42a787a44fd97579a46a246f27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 00:59:45.184143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ff4d6987-0707-4b01-ac20-ecebbd32b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 00:59:45.237792 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 958/1954] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 00:59:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 00:59:49.690257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 00:59:55.231254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=28 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:00:00.167655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb7401-5af3-4e25-99ec-4265de0e0025 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:00.168190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb7401-5af3-4e25-99ec-4265de0e0025 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:00.168190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deb7401-5af3-4e25-99ec-4265de0e0025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:00.187500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7deb7401-5af3-4e25-99ec-4265de0e0025 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:00.188719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7deb7401-5af3-4e25-99ec-4265de0e0025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:00.198882 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 997/1955] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 01:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:00:00.213172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d6987-0707-4b01-ac20-ecebbd32b53c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:00.226239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d6987-0707-4b01-ac20-ecebbd32b53c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:00.231792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:00.232428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:00.232709 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:00.232984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:00.240959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:00.243099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:00.253085 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 959/1956] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:04.484478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087e292-b345-4f1f-96b0-ae13ab8bc027 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:04.485006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087e292-b345-4f1f-96b0-ae13ab8bc027 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:04.485006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b087e292-b345-4f1f-96b0-ae13ab8bc027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:04.504142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b087e292-b345-4f1f-96b0-ae13ab8bc027 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:04.505396 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b087e292-b345-4f1f-96b0-ae13ab8bc027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:04.513040 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 998/1957] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 01:00:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:00:04.529891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:04.541506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24974c97-3bac-46d2-9daa-d3a0a93d2d36 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:04.546923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:04.547518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:04.547797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:04.548071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:04.556015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:04.557610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:04.567106 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 960/1958] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:08.581045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3afd-bda2-46cd-8f25-06edf29ea4de None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:08.581546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3afd-bda2-46cd-8f25-06edf29ea4de None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:08.581642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3afd-bda2-46cd-8f25-06edf29ea4de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:08.601121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f4fe3afd-bda2-46cd-8f25-06edf29ea4de None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:08.602335 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f4fe3afd-bda2-46cd-8f25-06edf29ea4de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:08.610584 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 999/1959] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 01:00:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:00:08.630599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:08.641924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d56e7f-ee94-431e-9917-f086a3aca6af service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:08.647882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:08.648530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:08.648807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:08.649075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:08.657377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:08.658868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:08.668529 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 961/1960] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:12.769803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1c2b3-0bed-4013-accc-04010425ec42 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:12.770306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1c2b3-0bed-4013-accc-04010425ec42 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:12.770407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a1c2b3-0bed-4013-accc-04010425ec42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:12.789450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-72a1c2b3-0bed-4013-accc-04010425ec42 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:12.790640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-72a1c2b3-0bed-4013-accc-04010425ec42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:12.798676 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1000/1961] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 01:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:00:12.813720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:12.825263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593acd78-88bb-4a2b-a2c4-f2d7047c52c0 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:12.830817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:12.831529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:12.831678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:12.831949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:12.840168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:12.841758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:12.850868 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 962/1962] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:00:16.958196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1288d-52f2-4ce0-b982-7f7d11791619 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:16.958678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1288d-52f2-4ce0-b982-7f7d11791619 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:16.958887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1288d-52f2-4ce0-b982-7f7d11791619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:16.978571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-61e1288d-52f2-4ce0-b982-7f7d11791619 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:16.979741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-61e1288d-52f2-4ce0-b982-7f7d11791619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:16.987465 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1001/1963] 10.222.0.43 () {58 vars in 1016 bytes} [Sun Oct 23 01:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:00:17.001526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:17.012630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2889776d-2332-4bb9-9b28-8e62daa25caf service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:17.018011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:17.018600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:17.018876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:17.019145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:17.027522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:17.029235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:17.038379 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 963/1964] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:18.981694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1288d-52f2-4ce0-b982-7f7d11791619 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:18.990962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1288d-52f2-4ce0-b982-7f7d11791619 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:18.995072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290525b1-c353-48f6-8fe0-56c36b5597dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:18.995667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290525b1-c353-48f6-8fe0-56c36b5597dd service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:18.996064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290525b1-c353-48f6-8fe0-56c36b5597dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:18.996221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290525b1-c353-48f6-8fe0-56c36b5597dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:18.998100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-290525b1-c353-48f6-8fe0-56c36b5597dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:19.011397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290525b1-c353-48f6-8fe0-56c36b5597dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:19.012838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290525b1-c353-48f6-8fe0-56c36b5597dd service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:19.023272 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1002/1965] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:20.353219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:20.362513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ca4cde-6bb6-4137-b9fb-0e072b563276 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:20.366052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:20.366643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:20.366924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:20.367248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:20.368968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:20.382057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:20.383486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:20.394012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 964/1966] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:00:29.018542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=21 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:00:30.389376 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=36 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:00:32.026010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11f329-3438-44d6-9400-23aa9a3cf0e8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:32.026579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11f329-3438-44d6-9400-23aa9a3cf0e8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:32.026579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11f329-3438-44d6-9400-23aa9a3cf0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:32.045141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-be11f329-3438-44d6-9400-23aa9a3cf0e8 None None] MFA Rules not processed for user `28563be0db564f8ebcf765a78970ce8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:32.046328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-be11f329-3438-44d6-9400-23aa9a3cf0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:32.100170 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1003/1967] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:00:32 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:00:42.093779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=25 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:00:45.083758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:45.095141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ade04ca-f0e5-4261-b64a-f3023d5f62b8 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:45.100565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a1b411-49d8-469b-9c81-7c3607abc0da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:45.101168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b411-49d8-469b-9c81-7c3607abc0da service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:45.101449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b411-49d8-469b-9c81-7c3607abc0da service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:45.101747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a1b411-49d8-469b-9c81-7c3607abc0da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:45.103406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d3a1b411-49d8-469b-9c81-7c3607abc0da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:45.115086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1b411-49d8-469b-9c81-7c3607abc0da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:45.116835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a1b411-49d8-469b-9c81-7c3607abc0da service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:45.126470 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 965/1968] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:53.651222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589430b-7511-4967-9c7f-b9e09a591874 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:53.651733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589430b-7511-4967-9c7f-b9e09a591874 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:53.652059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589430b-7511-4967-9c7f-b9e09a591874 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:53.654250 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1004/1969] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 01:00:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:00:53.663438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:53.663970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:53.663970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:53.684851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:53.686050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:53.694883 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 966/1970] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 01:00:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:00:53.710590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589430b-7511-4967-9c7f-b9e09a591874 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:53.725131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589430b-7511-4967-9c7f-b9e09a591874 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:53.731622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:53.733520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:53.734007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:53.734487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:53.737248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:53.739983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:53.744633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:53.747237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:53.757772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c4f3c-d069-4c66-9b06-c453aec43445 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:53.763311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:53.764398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:53.764683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:53.764969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] PATH_INFO: `/v3/users/4359f42a787a44fd97579a46a246f27a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:53.765190 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1005/1971] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:53.766257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4359f42a787a44fd97579a46a246f27a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:53.768202 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:53.821520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4359f42a787a44fd97579a46a246f27a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:53.822184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4359f42a787a44fd97579a46a246f27a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:53.830139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4359f42a787a44fd97579a46a246f27a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:53.841967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4359f42a787a44fd97579a46a246f27a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:53.842652 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4359f42a787a44fd97579a46a246f27a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:53.855437 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 967/1972] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:00:53 2022] DELETE /identity/v3/users/4359f42a787a44fd97579a46a246f27a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:00:54.021953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:54.032601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88fc44-5882-4ae4-a96b-37ab14b752e9 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:54.088620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:54.089246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:54.089524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:54.089797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] PATH_INFO: `/v3/projects/bd4636922d3f4936b68c2d38f53ab4d4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:54.090921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd4636922d3f4936b68c2d38f53ab4d4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:54.092840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:54.137797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd4636922d3f4936b68c2d38f53ab4d4', 'request_id': 'req-69af430d-6d35-4e8e-863b-25e8d7ee1c18'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:54.145520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd4636922d3f4936b68c2d38f53ab4d4', 'request_id': 'req-69af430d-6d35-4e8e-863b-25e8d7ee1c18'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:54.151526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd4636922d3f4936b68c2d38f53ab4d4', 'request_id': 'req-69af430d-6d35-4e8e-863b-25e8d7ee1c18'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:54.161593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] The token cache is being invalidate because project bd4636922d3f4936b68c2d38f53ab4d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:00:54.161873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:54.162496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-69af430d-6d35-4e8e-863b-25e8d7ee1c18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:00:54.163950 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1006/1973] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:00:54 2022] DELETE /identity/v3/projects/bd4636922d3f4936b68c2d38f53ab4d4 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:00:58.069135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:58.070838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:58.149981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a81a8-8ecd-40a5-acbe-a71fa9499c28 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:58.198613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:00:58.217535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:58.218317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:58.218620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:58.218892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:58.220763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:58.283423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:58.285161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:58.295290 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 968/1974] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:00:59.414510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc4d51-8a2c-426c-8995-42a8b254a685 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:59.415035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc4d51-8a2c-426c-8995-42a8b254a685 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:59.415141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc4d51-8a2c-426c-8995-42a8b254a685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:59.435327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b3cc4d51-8a2c-426c-8995-42a8b254a685 None None] MFA Rules not processed for user `959991a17c69459dbd665c2ae20095ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:00:59.436557 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b3cc4d51-8a2c-426c-8995-42a8b254a685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:59.493390 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1007/1975] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:00:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:00:59.804936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:00:59.806304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:59.869547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a7317-a715-4807-8f76-097616dde527 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:00:59.873452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affdaa35-a1ea-4330-a184-5a80dd68b955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:00:59.874059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdaa35-a1ea-4330-a184-5a80dd68b955 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:00:59.874460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdaa35-a1ea-4330-a184-5a80dd68b955 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:00:59.874938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affdaa35-a1ea-4330-a184-5a80dd68b955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:00:59.877925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-affdaa35-a1ea-4330-a184-5a80dd68b955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:00:59.890183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdaa35-a1ea-4330-a184-5a80dd68b955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:00:59.891710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affdaa35-a1ea-4330-a184-5a80dd68b955 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:00:59.899983 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 969/1976] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:00:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:00.285520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05164c6e-96e8-45d9-9818-e4e7c17b1393 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:00.285803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05164c6e-96e8-45d9-9818-e4e7c17b1393 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:00.286257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05164c6e-96e8-45d9-9818-e4e7c17b1393 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:00.287657 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1008/1977] 10.222.0.39 () {54 vars in 961 bytes} [Sun Oct 23 01:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:01:00.292696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:00.292980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:00.293815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:00.317600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] MFA Rules not processed for user `145f39f828664c56903ae5aeffa4dd52`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:01:00.318791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:00.382222 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 970/1978] 10.222.0.39 () {58 vars in 1058 bytes} [Sun Oct 23 01:01:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:01:00.390187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05164c6e-96e8-45d9-9818-e4e7c17b1393 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:00.403928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05164c6e-96e8-45d9-9818-e4e7c17b1393 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:00.408788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:00.409373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:00.409649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:00.409916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:00.412197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:00.423219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:00.424815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:00.434833 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1009/1979] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:02.937584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:02.939464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:02.969338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f02f3-d85c-47be-8032-0e812fa7d2b8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:02.981620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:02.982378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:02.982568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:02.982961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:02.986152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:03.004671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:03.006542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:03.011105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:03.014029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:03.029582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c853c973-b911-4fe8-a884-b722d8656001 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:03.033378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157b647-f98b-4ede-af59-3cbf2435eacb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:03.033942 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 971/1980] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:03.034717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157b647-f98b-4ede-af59-3cbf2435eacb service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:03.035066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157b647-f98b-4ede-af59-3cbf2435eacb service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:03.035398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157b647-f98b-4ede-af59-3cbf2435eacb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:03.038006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b157b647-f98b-4ede-af59-3cbf2435eacb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:03.047350 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:01:03.051647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157b647-f98b-4ede-af59-3cbf2435eacb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:03.053450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157b647-f98b-4ede-af59-3cbf2435eacb service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:03.068331 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1010/1981] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:03.761405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:03.782330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b2727-a29f-4b90-9161-284ae666cc8a service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:03.791169 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:03.791830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:03.792248 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:03.792571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:03.794844 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:03.813480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:03.813480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:03.819876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:01:03.827766 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 972/1982] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:06.633815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1bbc5-7842-4828-927d-718a7365281a None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:06.634323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1bbc5-7842-4828-927d-718a7365281a None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:06.634418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1bbc5-7842-4828-927d-718a7365281a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:06.653454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4bf1bbc5-7842-4828-927d-718a7365281a None None] MFA Rules not processed for user `11b0f829f5654010b2bd1a76888d7ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:01:06.654630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4bf1bbc5-7842-4828-927d-718a7365281a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:06.661373 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1011/1983] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:01:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:01:06.963233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:06.972440 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a05363f-4d56-4ab0-89a6-02e2639164c6 service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:06.975934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:06.976560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:06.976837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:06.977105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:06.978802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:06.989011 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:06.990413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:06.999510 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 973/1984] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:16.648689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:01:16.994881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:01:17.035536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1bbc5-7842-4828-927d-718a7365281a None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:17.049321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1bbc5-7842-4828-927d-718a7365281a None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:17.054914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6f3b7-380b-405c-9dc2-17f5f657b4f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:17.055529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f3b7-380b-405c-9dc2-17f5f657b4f7 service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:17.055808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f3b7-380b-405c-9dc2-17f5f657b4f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:17.056083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6f3b7-380b-405c-9dc2-17f5f657b4f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:17.057862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-01a6f3b7-380b-405c-9dc2-17f5f657b4f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:17.070150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6f3b7-380b-405c-9dc2-17f5f657b4f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:17.072043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6f3b7-380b-405c-9dc2-17f5f657b4f7 service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:17.082603 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1012/1985] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:27.078154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:01:37.774206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:37.775907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:37.832120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0778c22-63a5-4800-8d32-b156005da77f service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:37.835636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:37.836229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:37.836533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:37.836799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:37.838493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:37.896091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:37.897552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:37.906992 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 974/1986] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:43.506429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5f19-5d4d-4598-a855-79e514562934 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:43.506948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5f19-5d4d-4598-a855-79e514562934 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:43.506948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4f5f19-5d4d-4598-a855-79e514562934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:43.525400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8e4f5f19-5d4d-4598-a855-79e514562934 None None] MFA Rules not processed for user `992874ab1b9c4fce90b46e659910d512`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:01:43.526584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e4f5f19-5d4d-4598-a855-79e514562934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:43.533356 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1013/1987] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:01:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:01:44.842815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:01:44.851302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817bec31-a104-48fa-9280-bce1b3577ef9 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:01:44.855035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:01:44.856400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:01:44.856691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:01:44.856970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:01:44.858718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:01:44.916975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:01:44.919032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:01:44.927832 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 975/1988] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:01:44 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:01:53.520623 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:01:54.923833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=76 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:05.624057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f5f19-5d4d-4598-a855-79e514562934 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:05.633705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4f5f19-5d4d-4598-a855-79e514562934 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:05.637167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:05.637760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:05.638032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:05.638299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:05.639967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:05.701116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:05.702531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:05.712005 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1014/1989] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:08.615451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:08.617023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:08.632333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1c6f13-b0da-495d-ad11-c44f40eba3e1 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:08.637601 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:08.638331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:08.638703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:08.638999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] PATH_INFO: `/v3/users/28563be0db564f8ebcf765a78970ce8f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:08.640097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28563be0db564f8ebcf765a78970ce8f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:08.642118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:08.688525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28563be0db564f8ebcf765a78970ce8f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:08.689193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28563be0db564f8ebcf765a78970ce8f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:08.696032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28563be0db564f8ebcf765a78970ce8f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:08.707542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28563be0db564f8ebcf765a78970ce8f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:08.708170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-550a68c3-e7ef-49ed-85c8-8e6109fed8e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28563be0db564f8ebcf765a78970ce8f'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:08.721131 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 976/1990] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:08 2022] DELETE /identity/v3/users/28563be0db564f8ebcf765a78970ce8f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:08.975509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:08.986421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc8171d-35ea-4eda-9fc2-32ae3dd46459 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.042899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.043530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.043817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.044086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] PATH_INFO: `/v3/projects/ebac7696faf14dfea4957e97b589029d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.045275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebac7696faf14dfea4957e97b589029d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.047172 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.090900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebac7696faf14dfea4957e97b589029d', 'request_id': 'req-e9c21003-b9fb-4012-b820-d492edce7f08'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:09.098197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebac7696faf14dfea4957e97b589029d', 'request_id': 'req-e9c21003-b9fb-4012-b820-d492edce7f08'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:09.104322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebac7696faf14dfea4957e97b589029d', 'request_id': 'req-e9c21003-b9fb-4012-b820-d492edce7f08'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:09.114863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] The token cache is being invalidate because project ebac7696faf14dfea4957e97b589029d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:09.115154 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:09.115771 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e9c21003-b9fb-4012-b820-d492edce7f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:09.117257 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1015/1991] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:08 2022] DELETE /identity/v3/projects/ebac7696faf14dfea4957e97b589029d => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:09.122253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d375-40ec-4d1d-8485-fdc3e515c185 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.122539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d375-40ec-4d1d-8485-fdc3e515c185 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.122851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da7d375-40ec-4d1d-8485-fdc3e515c185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.142452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0da7d375-40ec-4d1d-8485-fdc3e515c185 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:09.143641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0da7d375-40ec-4d1d-8485-fdc3e515c185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:09.201977 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 977/1992] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:09.209736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1da84-b1b9-4367-8001-5e7063dea764 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.210030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1da84-b1b9-4367-8001-5e7063dea764 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.210301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1da84-b1b9-4367-8001-5e7063dea764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.230603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-48d1da84-b1b9-4367-8001-5e7063dea764 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:09.231779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-48d1da84-b1b9-4367-8001-5e7063dea764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:09.239474 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1016/1993] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:09.243663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7d375-40ec-4d1d-8485-fdc3e515c185 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:09.254082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da7d375-40ec-4d1d-8485-fdc3e515c185 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.258650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.259246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.259521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.259789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.260465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:09.260789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.262909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.274228 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 978/1994] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:02:09.278619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1da84-b1b9-4367-8001-5e7063dea764 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:09.287775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1da84-b1b9-4367-8001-5e7063dea764 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.292401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.293023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.293313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.293565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.294204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.296091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.312690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:09.326463 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1017/1995] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:09 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:09.330547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:09.340689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cded9-6b99-4666-a509-01c38845e654 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.395269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.395880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.396155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.396454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.397069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.398955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.406711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:09.421700 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 979/1996] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:09 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:09.425425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:09.436548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6790d9f-8ea9-4646-9e9b-96be0125b7fc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.441120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.441717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.441994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.442263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.442897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.444515 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.454474 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1018/1997] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:02:09.458263 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:09.467993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f6486a-0328-4c70-a4b9-74620f8a4729 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.472542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b6de24-0f67-4e5f-b865-377461e27109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.473130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6de24-0f67-4e5f-b865-377461e27109 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.473407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6de24-0f67-4e5f-b865-377461e27109 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.473678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b6de24-0f67-4e5f-b865-377461e27109 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.474310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6de24-0f67-4e5f-b865-377461e27109 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.475905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b6de24-0f67-4e5f-b865-377461e27109 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.487216 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 980/1998] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:09.490890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:09.501152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694dc07-0777-494a-bd82-5a78bb733bfc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.505953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.506594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.506871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.507139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] PATH_INFO: `/v3/projects/09d2b71a13854907af2b7b235c0a39ce/users/6bbfa8686c3b411b9a164daf8f9786bb/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.521273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09d2b71a13854907af2b7b235c0a39ce, user_id=6bbfa8686c3b411b9a164daf8f9786bb, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.523884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.536989 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1019/1999] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:09 2022] PUT /identity/v3/projects/09d2b71a13854907af2b7b235c0a39ce/users/6bbfa8686c3b411b9a164daf8f9786bb/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:09.541969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33046-273e-4163-bc47-7371778db815 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.542771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33046-273e-4163-bc47-7371778db815 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.543042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a33046-273e-4163-bc47-7371778db815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.562071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-63a33046-273e-4163-bc47-7371778db815 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:09.563254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-63a33046-273e-4163-bc47-7371778db815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:09.619525 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 981/2000] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:02:09.630138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:09.631784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:09.686729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e61451-4895-43c0-bc8e-a6ae1baef697 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:09.690318 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:09.690982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:09.691270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:09.691537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:09.700618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:09.702054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:09.711757 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1020/2001] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:10.345123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146cb78-d7e1-4c8a-8045-4c7683df4d95 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:10.345415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146cb78-d7e1-4c8a-8045-4c7683df4d95 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:10.345687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146cb78-d7e1-4c8a-8045-4c7683df4d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:10.388515 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c146cb78-d7e1-4c8a-8045-4c7683df4d95 None None] MFA Rules not processed for user `6bbfa8686c3b411b9a164daf8f9786bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:10.389716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c146cb78-d7e1-4c8a-8045-4c7683df4d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:10.455273 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 982/2002] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:10 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:10.459754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:10.469847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac809a5-599b-4d37-82e2-8e39985a4a6a service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:10.474431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:10.475028 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:10.475305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:10.475574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:10.476262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:10.478148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:10.509144 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1021/2003] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:10 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:10.510247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146cb78-d7e1-4c8a-8045-4c7683df4d95 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:10.521706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c146cb78-d7e1-4c8a-8045-4c7683df4d95 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:10.577536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:10.578213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:10.578555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:10.578897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:10.579584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:10.581683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:10.597721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:10.608289 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 983/2004] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:10 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:10.609466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:10.619505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e03c80-0217-43bc-9815-5c4cc9fca935 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:10.624036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:10.624671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:10.624950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:10.625222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:10.625862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:10.627427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:10.638004 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1022/2005] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:10.642027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:10.652293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cef399-ec7a-493a-ab65-169d5b5aef8b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:10.656788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:10.657428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:10.657699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:10.657964 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] PATH_INFO: `/v3/projects/7036246d5f7843608f9f5aaf15ea9b20/users/462e1189971d46f2be897270780c45a4/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:10.672679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7036246d5f7843608f9f5aaf15ea9b20, user_id=462e1189971d46f2be897270780c45a4, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:10.675298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:10.687245 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 984/2006] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:10 2022] PUT /identity/v3/projects/7036246d5f7843608f9f5aaf15ea9b20/users/462e1189971d46f2be897270780c45a4/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:10.691322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:10.701078 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637be956-51aa-4944-a22d-477f315ff1f1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:10.755710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1094b-9355-4d03-a296-7df6cbc61d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:10.756240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1094b-9355-4d03-a296-7df6cbc61d8d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:10.756540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1094b-9355-4d03-a296-7df6cbc61d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:10.756813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1094b-9355-4d03-a296-7df6cbc61d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:10.757441 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1094b-9355-4d03-a296-7df6cbc61d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:10.759019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1094b-9355-4d03-a296-7df6cbc61d8d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:10.768455 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1023/2007] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:10.772667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:10.783057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d603d7-5b10-485e-a087-e68012884a08 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:10.787531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:10.788171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:10.788563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:10.788833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] PATH_INFO: `/v3/projects/7036246d5f7843608f9f5aaf15ea9b20/users/462e1189971d46f2be897270780c45a4/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:10.790674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7036246d5f7843608f9f5aaf15ea9b20, user_id=462e1189971d46f2be897270780c45a4, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:10.793922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:10.803967 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 985/2008] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:10 2022] PUT /identity/v3/projects/7036246d5f7843608f9f5aaf15ea9b20/users/462e1189971d46f2be897270780c45a4/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:11.659668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4a94-1b47-415a-a34b-900d1ee3009c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:11.660182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4a94-1b47-415a-a34b-900d1ee3009c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:11.660279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4a94-1b47-415a-a34b-900d1ee3009c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:11.689454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a9da4a94-1b47-415a-a34b-900d1ee3009c None None] MFA Rules not processed for user `462e1189971d46f2be897270780c45a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:11.690648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a9da4a94-1b47-415a-a34b-900d1ee3009c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:11.703008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:11.761511 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1024/2009] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:11 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:12.846908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:12.909146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d4fdd6-6c5a-4846-8f87-681d65ed6e87 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:12.913636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:12.914258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:12.914533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:12.914799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] PATH_INFO: `/v3/users/6bbfa8686c3b411b9a164daf8f9786bb` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:12.915889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bbfa8686c3b411b9a164daf8f9786bb)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:12.918458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:12.962821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bbfa8686c3b411b9a164daf8f9786bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:12.963466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bbfa8686c3b411b9a164daf8f9786bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:12.970981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bbfa8686c3b411b9a164daf8f9786bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:12.983319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bbfa8686c3b411b9a164daf8f9786bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:12.983947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bbfa8686c3b411b9a164daf8f9786bb'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:12.996812 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 986/2010] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:12 2022] DELETE /identity/v3/users/6bbfa8686c3b411b9a164daf8f9786bb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:13.166190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4a94-1b47-415a-a34b-900d1ee3009c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:13.178013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4a94-1b47-415a-a34b-900d1ee3009c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:13.233987 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:13.234605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:13.234879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:13.235142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] PATH_INFO: `/v3/projects/09d2b71a13854907af2b7b235c0a39ce` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:13.236226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09d2b71a13854907af2b7b235c0a39ce)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:13.238191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:13.283033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09d2b71a13854907af2b7b235c0a39ce', 'request_id': 'req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.289998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09d2b71a13854907af2b7b235c0a39ce', 'request_id': 'req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.296610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09d2b71a13854907af2b7b235c0a39ce', 'request_id': 'req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.306530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] The token cache is being invalidate because project 09d2b71a13854907af2b7b235c0a39ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:13.306808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.307413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.309444 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1025/2011] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:13 2022] DELETE /identity/v3/projects/09d2b71a13854907af2b7b235c0a39ce => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:13.313420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:13.314902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:13.327477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de927c74-89d6-47c3-9db8-d4bc64c8c3a1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:13.364550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:13.384442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:13.385068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:13.385347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:13.385619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] PATH_INFO: `/v3/users/462e1189971d46f2be897270780c45a4` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:13.386737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462e1189971d46f2be897270780c45a4)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:13.388703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:13.434131 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462e1189971d46f2be897270780c45a4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.434767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462e1189971d46f2be897270780c45a4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.441752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462e1189971d46f2be897270780c45a4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.453552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462e1189971d46f2be897270780c45a4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.454220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0c5c09d4-46c1-4764-a3c2-67b6b8c6f5e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462e1189971d46f2be897270780c45a4'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.467474 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 987/2012] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:13 2022] DELETE /identity/v3/users/462e1189971d46f2be897270780c45a4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:02:13.638107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:13.639538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:13.652970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f6173-670b-4c53-aa7d-cb3a82d917c6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:13.694164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:13.715184 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:13.715816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:13.716092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:13.716397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] PATH_INFO: `/v3/projects/7036246d5f7843608f9f5aaf15ea9b20` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:13.717662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7036246d5f7843608f9f5aaf15ea9b20)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:13.719706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:13.767246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7036246d5f7843608f9f5aaf15ea9b20', 'request_id': 'req-84ee1f17-52e2-4b1a-b00c-84856c530993'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.775003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7036246d5f7843608f9f5aaf15ea9b20', 'request_id': 'req-84ee1f17-52e2-4b1a-b00c-84856c530993'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.782204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7036246d5f7843608f9f5aaf15ea9b20', 'request_id': 'req-84ee1f17-52e2-4b1a-b00c-84856c530993'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.793461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] The token cache is being invalidate because project 7036246d5f7843608f9f5aaf15ea9b20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:13.793743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.794380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-84ee1f17-52e2-4b1a-b00c-84856c530993 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:13.795829 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1026/2013] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:13 2022] DELETE /identity/v3/projects/7036246d5f7843608f9f5aaf15ea9b20 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:13.800691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab50f6-0324-4d54-81f7-3a99cf91c63d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:13.800974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab50f6-0324-4d54-81f7-3a99cf91c63d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:13.801264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab50f6-0324-4d54-81f7-3a99cf91c63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:13.820641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-86ab50f6-0324-4d54-81f7-3a99cf91c63d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:13.821813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-86ab50f6-0324-4d54-81f7-3a99cf91c63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:13.879661 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 988/2014] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:13.887747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f142aa-a6f1-4ae6-83fa-6a1ac09ba2f2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:13.888060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f142aa-a6f1-4ae6-83fa-6a1ac09ba2f2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:13.888372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f142aa-a6f1-4ae6-83fa-6a1ac09ba2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:13.908405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f0f142aa-a6f1-4ae6-83fa-6a1ac09ba2f2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:13.909691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f0f142aa-a6f1-4ae6-83fa-6a1ac09ba2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:13.917480 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1027/2015] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:13.921512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab50f6-0324-4d54-81f7-3a99cf91c63d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:13.930849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab50f6-0324-4d54-81f7-3a99cf91c63d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:13.935289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:13.935880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:13.936155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:13.936457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:13.937095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:13.937392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:13.938954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:13.950576 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 989/2016] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:02:13.954802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f142aa-a6f1-4ae6-83fa-6a1ac09ba2f2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:13.964478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f142aa-a6f1-4ae6-83fa-6a1ac09ba2f2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:13.969093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:13.969681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:13.969959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:13.970228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:13.970861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:13.972787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:13.999748 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1028/2017] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:13 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:14.004234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.014763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189155f3-c0ce-462e-ba3a-6ee474d74701 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.057143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:14.072103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.072735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.073037 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.073993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.074623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.076574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.098187 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 990/2018] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:14 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:14.102278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.112372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f90b5-afce-44e2-b234-2f5ddb37ce06 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.117261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.117880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.118156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.118422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.119060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.120885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.130942 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1029/2019] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:14.134718 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.144977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9681f4-9827-46c7-bd92-5869ba088bfd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.149587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af1b11-176f-436d-88a7-0b79795f4e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.150176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af1b11-176f-436d-88a7-0b79795f4e41 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.150456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af1b11-176f-436d-88a7-0b79795f4e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.150728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af1b11-176f-436d-88a7-0b79795f4e41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.152000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af1b11-176f-436d-88a7-0b79795f4e41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.153604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af1b11-176f-436d-88a7-0b79795f4e41 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.165114 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 991/2020] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:14.169107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.178789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4ae31a-f7dd-42e9-9341-e42f0eb971d2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.183338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.183970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.184243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.184540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] PATH_INFO: `/v3/projects/ca48d809f24b46e8b96b2dbf6cff339d/users/c194eccf14b44ec385f25308c1037c17/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.199007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca48d809f24b46e8b96b2dbf6cff339d, user_id=c194eccf14b44ec385f25308c1037c17, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.201659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.214428 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1030/2021] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:14 2022] PUT /identity/v3/projects/ca48d809f24b46e8b96b2dbf6cff339d/users/c194eccf14b44ec385f25308c1037c17/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:14.225579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03908f53-3804-4707-ae1c-d47ff574bdc1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.225863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03908f53-3804-4707-ae1c-d47ff574bdc1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.226136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03908f53-3804-4707-ae1c-d47ff574bdc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.254070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-03908f53-3804-4707-ae1c-d47ff574bdc1 None None] MFA Rules not processed for user `c194eccf14b44ec385f25308c1037c17`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:14.255255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-03908f53-3804-4707-ae1c-d47ff574bdc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:14.319927 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 992/2022] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:14.325032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.384126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f09c5f-87f9-436a-b53d-eba25acad53f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.389215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.389808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.390085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.390353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.390975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.392898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.406433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:14.419822 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1031/2023] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:14 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:14.423776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03908f53-3804-4707-ae1c-d47ff574bdc1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.434721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03908f53-3804-4707-ae1c-d47ff574bdc1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.488609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.489205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.489480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.489744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.490371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.492247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.514029 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 993/2024] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:14 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:14.517971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.528144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43571cf1-869d-46ca-8f69-6991768b879f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.532748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.533340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.533618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.533891 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.534529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.536115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.546983 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1032/2025] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:14.550977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.561212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfcdc1-42b8-4faf-9e4b-835f447a23c9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.565691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.566356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.566630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.566894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] PATH_INFO: `/v3/projects/5f74e64535e746aa85abe9bf9741301a/users/76c2367bdcb146e2bcebb58cc50e64c2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.582375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f74e64535e746aa85abe9bf9741301a, user_id=76c2367bdcb146e2bcebb58cc50e64c2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.585029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.597104 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 994/2026] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:14 2022] PUT /identity/v3/projects/5f74e64535e746aa85abe9bf9741301a/users/76c2367bdcb146e2bcebb58cc50e64c2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:14.601079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.610932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5742b0b5-3cd6-4696-ba20-0f11a3ea95a7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.666124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41835e4a-f8ce-48dd-862b-677189b83c27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.666650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835e4a-f8ce-48dd-862b-677189b83c27 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.667459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835e4a-f8ce-48dd-862b-677189b83c27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.667729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41835e4a-f8ce-48dd-862b-677189b83c27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.668393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41835e4a-f8ce-48dd-862b-677189b83c27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.669966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41835e4a-f8ce-48dd-862b-677189b83c27 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.679242 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1033/2027] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:14.683846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.694140 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18d918f-518d-41ca-a4e1-6f924934fa97 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.698801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.699451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.699733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.700006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] PATH_INFO: `/v3/projects/5f74e64535e746aa85abe9bf9741301a/users/76c2367bdcb146e2bcebb58cc50e64c2/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.701933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f74e64535e746aa85abe9bf9741301a, user_id=76c2367bdcb146e2bcebb58cc50e64c2, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.704608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.714587 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 995/2028] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:14 2022] PUT /identity/v3/projects/5f74e64535e746aa85abe9bf9741301a/users/76c2367bdcb146e2bcebb58cc50e64c2/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:02:14.724544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7c31c-aa43-4922-87f2-e9c5d631aa71 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.724942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7c31c-aa43-4922-87f2-e9c5d631aa71 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.725227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a7c31c-aa43-4922-87f2-e9c5d631aa71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.753423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b5a7c31c-aa43-4922-87f2-e9c5d631aa71 None None] MFA Rules not processed for user `76c2367bdcb146e2bcebb58cc50e64c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:14.754612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b5a7c31c-aa43-4922-87f2-e9c5d631aa71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:14.823758 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1034/2029] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:14.836551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:14.838251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:14.848655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:14.899136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e753-c077-4266-aad3-3e2c3f850f55 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:14.903452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc5aef-f489-413e-8ad6-0e72d0aea48f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:14.904060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5aef-f489-413e-8ad6-0e72d0aea48f service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:14.904366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5aef-f489-413e-8ad6-0e72d0aea48f service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:14.904640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc5aef-f489-413e-8ad6-0e72d0aea48f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:14.913970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc5aef-f489-413e-8ad6-0e72d0aea48f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:14.915380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fc5aef-f489-413e-8ad6-0e72d0aea48f service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:14.925451 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 996/2030] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:15.003736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7c31c-aa43-4922-87f2-e9c5d631aa71 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:15.013626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a7c31c-aa43-4922-87f2-e9c5d631aa71 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:15.017171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578b1a87-710c-4d5a-859d-6a212b073ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:15.017759 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b1a87-710c-4d5a-859d-6a212b073ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:15.018031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b1a87-710c-4d5a-859d-6a212b073ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:15.018297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578b1a87-710c-4d5a-859d-6a212b073ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:15.073953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b1a87-710c-4d5a-859d-6a212b073ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:15.075415 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578b1a87-710c-4d5a-859d-6a212b073ef2 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:15.084091 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1035/2031] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:15 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:18.641387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7f5e-d07e-4186-8047-caa3ea86d53b None None] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:18.641928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7f5e-d07e-4186-8047-caa3ea86d53b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:18.642096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ac7f5e-d07e-4186-8047-caa3ea86d53b None None] PATH_INFO: `/` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:18.643628 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 997/2032] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 01:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:02:18.649792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce00977-c6f6-4852-8396-644cf2e6488d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:18.650314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce00977-c6f6-4852-8396-644cf2e6488d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:18.650412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce00977-c6f6-4852-8396-644cf2e6488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:18.657997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:18.671797 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bce00977-c6f6-4852-8396-644cf2e6488d None None] MFA Rules not processed for user `bb5f82d3cfc14eacbe4cc40bcd0dfb20`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:18.673043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bce00977-c6f6-4852-8396-644cf2e6488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:18.724622 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1036/2033] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 01:02:18 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:18.737527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7f5e-d07e-4186-8047-caa3ea86d53b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:18.739839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e6ac7f5e-d07e-4186-8047-caa3ea86d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:18.754430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ac7f5e-d07e-4186-8047-caa3ea86d53b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:18.757990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091ae9d-41e5-47b9-8a4f-41e8e5ed38c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:18.758617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091ae9d-41e5-47b9-8a4f-41e8e5ed38c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:18.758889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091ae9d-41e5-47b9-8a4f-41e8e5ed38c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:18.759149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091ae9d-41e5-47b9-8a4f-41e8e5ed38c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:18.766910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091ae9d-41e5-47b9-8a4f-41e8e5ed38c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:18.772063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091ae9d-41e5-47b9-8a4f-41e8e5ed38c3 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:18.780022 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 998/2034] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:18.861591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cceb-5e13-4b31-a279-2f87fc3d59a5 None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:18.861875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cceb-5e13-4b31-a279-2f87fc3d59a5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:18.862152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4cceb-5e13-4b31-a279-2f87fc3d59a5 None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:18.863544 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1037/2035] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 01:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:02:18.869481 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea7ed3-8d40-4888-9c28-4072a43c60d7 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:18.870533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea7ed3-8d40-4888-9c28-4072a43c60d7 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:18.870805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ea7ed3-8d40-4888-9c28-4072a43c60d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:18.890905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-d9ea7ed3-8d40-4888-9c28-4072a43c60d7 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:18.892102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d9ea7ed3-8d40-4888-9c28-4072a43c60d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:18.962102 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 999/2036] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 01:02:18 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:18.973246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cceb-5e13-4b31-a279-2f87fc3d59a5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:18.975235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7de4cceb-5e13-4b31-a279-2f87fc3d59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:18.988924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4cceb-5e13-4b31-a279-2f87fc3d59a5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:18.992757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647ae96-fba6-4dc0-a06a-cf00e50dffb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:18.993385 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647ae96-fba6-4dc0-a06a-cf00e50dffb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:18.993663 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647ae96-fba6-4dc0-a06a-cf00e50dffb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:18.993934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647ae96-fba6-4dc0-a06a-cf00e50dffb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:19.005288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647ae96-fba6-4dc0-a06a-cf00e50dffb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:19.006827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647ae96-fba6-4dc0-a06a-cf00e50dffb7 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:19.018181 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1038/2037] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:21.878368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea7ed3-8d40-4888-9c28-4072a43c60d7 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:21.939690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ea7ed3-8d40-4888-9c28-4072a43c60d7 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:21.944171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:21.944834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:21.945141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:21.945413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] PATH_INFO: `/v3/users/c194eccf14b44ec385f25308c1037c17` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:21.946535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c194eccf14b44ec385f25308c1037c17)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:21.948478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:21.979272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:21.994017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c194eccf14b44ec385f25308c1037c17'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:21.994666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c194eccf14b44ec385f25308c1037c17'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.002030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c194eccf14b44ec385f25308c1037c17'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.013932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c194eccf14b44ec385f25308c1037c17'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.014558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c194eccf14b44ec385f25308c1037c17'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.027016 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1000/2038] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:21 2022] DELETE /identity/v3/users/c194eccf14b44ec385f25308c1037c17 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:22.031520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:22.032000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:22.032096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:22.051531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:22.052849 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:22.111449 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1039/2039] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:22.121742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:22.123114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:22.181319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf84a00-b368-45e1-a4c0-be172a69b478 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:22.184927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:22.185591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:22.185864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:22.186132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:22.194978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:22.196422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:22.205835 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1001/2040] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:22.476323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:22.477835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:22.492263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a597e1-f06a-4c0c-bb9b-7b31dc55c772 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:22.497638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:22.498445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:22.498898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:22.499279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] PATH_INFO: `/v3/projects/ca48d809f24b46e8b96b2dbf6cff339d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:22.500536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca48d809f24b46e8b96b2dbf6cff339d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:22.502428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:22.547377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca48d809f24b46e8b96b2dbf6cff339d', 'request_id': 'req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.554457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca48d809f24b46e8b96b2dbf6cff339d', 'request_id': 'req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.560553 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca48d809f24b46e8b96b2dbf6cff339d', 'request_id': 'req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.570450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] The token cache is being invalidate because project ca48d809f24b46e8b96b2dbf6cff339d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:22.570728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.571337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.572840 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1040/2041] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:22 2022] DELETE /identity/v3/projects/ca48d809f24b46e8b96b2dbf6cff339d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:22.577274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:22.586756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40d5def-67f6-4048-93e6-32842db90ac3 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:22.641993 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:22.642611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:22.642891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:22.643159 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] PATH_INFO: `/v3/users/76c2367bdcb146e2bcebb58cc50e64c2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:22.644252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c2367bdcb146e2bcebb58cc50e64c2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:22.646199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:22.689434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c2367bdcb146e2bcebb58cc50e64c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.690074 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c2367bdcb146e2bcebb58cc50e64c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.694385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:22.699162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c2367bdcb146e2bcebb58cc50e64c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.711966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c2367bdcb146e2bcebb58cc50e64c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.712600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aab83bdb-ca9e-48f1-9f68-14a37a8589e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c2367bdcb146e2bcebb58cc50e64c2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:22.725448 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1002/2042] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:22 2022] DELETE /identity/v3/users/76c2367bdcb146e2bcebb58cc50e64c2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:22.991859 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:22.993466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:23.007494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66388ea9-a74c-48e1-863f-2cb8b01ccd1b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.026684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:23.066322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.066938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.067169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.067438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] PATH_INFO: `/v3/projects/5f74e64535e746aa85abe9bf9741301a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.068573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f74e64535e746aa85abe9bf9741301a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.070474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.115030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f74e64535e746aa85abe9bf9741301a', 'request_id': 'req-3e894d53-0c26-4b20-ac34-6b5607b16a3c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:23.122355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f74e64535e746aa85abe9bf9741301a', 'request_id': 'req-3e894d53-0c26-4b20-ac34-6b5607b16a3c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:23.129033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f74e64535e746aa85abe9bf9741301a', 'request_id': 'req-3e894d53-0c26-4b20-ac34-6b5607b16a3c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:23.140985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] The token cache is being invalidate because project 5f74e64535e746aa85abe9bf9741301a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:23.141270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:23.141905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e894d53-0c26-4b20-ac34-6b5607b16a3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:23.143381 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1041/2043] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:22 2022] DELETE /identity/v3/projects/5f74e64535e746aa85abe9bf9741301a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:23.148569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31a55-8da8-4c3d-9a32-00bfb6e5dfe9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.149033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31a55-8da8-4c3d-9a32-00bfb6e5dfe9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.149243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd31a55-8da8-4c3d-9a32-00bfb6e5dfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.170690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fdd31a55-8da8-4c3d-9a32-00bfb6e5dfe9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:23.171885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fdd31a55-8da8-4c3d-9a32-00bfb6e5dfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:23.231501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1003/2044] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:02:23.239519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8c0bf-fe98-4d23-8ef9-3b36b9b10905 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.239807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8c0bf-fe98-4d23-8ef9-3b36b9b10905 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.240083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8c0bf-fe98-4d23-8ef9-3b36b9b10905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.260035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-65e8c0bf-fe98-4d23-8ef9-3b36b9b10905 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:23.261869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-65e8c0bf-fe98-4d23-8ef9-3b36b9b10905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:23.269568 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1042/2045] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:23.273812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd31a55-8da8-4c3d-9a32-00bfb6e5dfe9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.283733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd31a55-8da8-4c3d-9a32-00bfb6e5dfe9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.288560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.289176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.289456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.289732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.290377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:23.290816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.292961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.304850 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1004/2046] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:02:23.309877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8c0bf-fe98-4d23-8ef9-3b36b9b10905 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.319131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8c0bf-fe98-4d23-8ef9-3b36b9b10905 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.324102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.324774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.325053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.325340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.325973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.327889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.362007 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1043/2047] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:23 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:23.362433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.375236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10128cdf-c740-4f5f-bc9a-ceea1800adf6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.435152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.435819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.436027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.436293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.436957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.438854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.463034 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1005/2048] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:23 2022] POST /identity/v3/users => generated 282 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:23.464431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.474717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c1689-0e8e-4924-afdc-99a714e9f33a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.480012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.480718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.481068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.481412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.482126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.483880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.493976 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1044/2049] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:23.498557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.509067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc79d7a-ab2a-4cce-89c1-341773b90d52 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.513647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e626e3-c591-427b-a80c-dfcfb8b01d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.514381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e626e3-c591-427b-a80c-dfcfb8b01d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.514658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e626e3-c591-427b-a80c-dfcfb8b01d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.514921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e626e3-c591-427b-a80c-dfcfb8b01d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.515558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e626e3-c591-427b-a80c-dfcfb8b01d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.517186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e626e3-c591-427b-a80c-dfcfb8b01d86 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.528613 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1006/2050] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:23.532763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.543461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad10fab7-29c2-41d3-987d-e7c3836d7e98 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.548057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.548745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.549026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.549305 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] PATH_INFO: `/v3/projects/ceb717ba574f4d65be22d6bf8f4897e8/users/391c78ff23ae442284b0aa8b2403b256/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.564143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb717ba574f4d65be22d6bf8f4897e8, user_id=391c78ff23ae442284b0aa8b2403b256, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.566824 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.579726 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1045/2051] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:23 2022] PUT /identity/v3/projects/ceb717ba574f4d65be22d6bf8f4897e8/users/391c78ff23ae442284b0aa8b2403b256/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:23.591635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024f4dc-12b3-4438-8b47-61394575e21b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.591915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024f4dc-12b3-4438-8b47-61394575e21b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.592184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7024f4dc-12b3-4438-8b47-61394575e21b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.622343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7024f4dc-12b3-4438-8b47-61394575e21b None None] MFA Rules not processed for user `391c78ff23ae442284b0aa8b2403b256`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:23.623531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7024f4dc-12b3-4438-8b47-61394575e21b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:23.646528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:23.692212 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1007/2052] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:23.696632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.746682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:23.762417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff530f-8051-4c9e-a9e2-72a9626aa7ba admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.767702 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.768348 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.770888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.771169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.771831 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.773844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.805778 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1046/2053] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:23 2022] POST /identity/v3/projects => generated 363 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:23.809787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024f4dc-12b3-4438-8b47-61394575e21b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.822292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7024f4dc-12b3-4438-8b47-61394575e21b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.885127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.885896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.886171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.886439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.887076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.889014 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.911461 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1008/2054] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:23 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:23.915444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.926480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e9eab-c627-416f-887e-bc67aa34e197 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.931657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.932346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.932721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.933051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.934072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.935530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:23.949182 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1047/2055] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:23.950400 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:23.960325 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff98f-ed21-413b-8bb2-1f25f4cb8133 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:23.965584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:23.966254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:23.966529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:23.966798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] PATH_INFO: `/v3/projects/39b94e63060d4a2a84d658f6effb1345/users/ae9877df15354fb69e7282148b89bab0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:23.982451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b94e63060d4a2a84d658f6effb1345, user_id=ae9877df15354fb69e7282148b89bab0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:23.985155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:24.000559 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1009/2056] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:23 2022] PUT /identity/v3/projects/39b94e63060d4a2a84d658f6effb1345/users/ae9877df15354fb69e7282148b89bab0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:02:24.004682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:24.015341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6e959-23b2-41b3-b67c-1b5a5c2f9a72 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:24.070208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853b543-1b78-4144-a1b4-59bae5ece453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:24.070836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853b543-1b78-4144-a1b4-59bae5ece453 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:24.071065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853b543-1b78-4144-a1b4-59bae5ece453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:24.071333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853b543-1b78-4144-a1b4-59bae5ece453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:24.071963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853b543-1b78-4144-a1b4-59bae5ece453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:24.074076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853b543-1b78-4144-a1b4-59bae5ece453 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:24.083508 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1048/2057] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:02:24.087574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:24.098095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045ddc2-0e74-441f-9c01-b58657b7173d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:24.102837 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:24.103480 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:24.103753 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:24.104021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] PATH_INFO: `/v3/projects/39b94e63060d4a2a84d658f6effb1345/users/ae9877df15354fb69e7282148b89bab0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:24.105956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b94e63060d4a2a84d658f6effb1345, user_id=ae9877df15354fb69e7282148b89bab0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:24.108708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:24.118929 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1010/2058] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:24 2022] PUT /identity/v3/projects/39b94e63060d4a2a84d658f6effb1345/users/ae9877df15354fb69e7282148b89bab0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:24.129384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17602b-d6bb-4198-ac33-48bd4edd40ac None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:24.129675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17602b-d6bb-4198-ac33-48bd4edd40ac None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:24.129954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17602b-d6bb-4198-ac33-48bd4edd40ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:24.158008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bb17602b-d6bb-4198-ac33-48bd4edd40ac None None] MFA Rules not processed for user `ae9877df15354fb69e7282148b89bab0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:24.159203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bb17602b-d6bb-4198-ac33-48bd4edd40ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:24.225777 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1049/2059] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:24.236714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:24.238476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:24.295526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef652f9a-d679-4203-9c28-3a4f84e7b481 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:24.299165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:24.299761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:24.300040 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:24.300323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:24.309139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:24.310553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:24.319942 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1011/2060] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:24.396232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17602b-d6bb-4198-ac33-48bd4edd40ac None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:24.405790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17602b-d6bb-4198-ac33-48bd4edd40ac None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:24.409382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d59dcc0-c7a2-4006-933b-3caf7636a654 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:24.409974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dcc0-c7a2-4006-933b-3caf7636a654 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:24.410250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dcc0-c7a2-4006-933b-3caf7636a654 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:24.410519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d59dcc0-c7a2-4006-933b-3caf7636a654 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:24.453529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:24.469409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59dcc0-c7a2-4006-933b-3caf7636a654 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:24.470826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d59dcc0-c7a2-4006-933b-3caf7636a654 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:24.479517 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1050/2061] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:31.514924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:31.534898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:31.577950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616da792-4bb9-4e99-b659-1e077e8f26c1 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:31.583352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:31.584008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:31.584286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:31.584587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] PATH_INFO: `/v3/users/391c78ff23ae442284b0aa8b2403b256` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:31.585729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=391c78ff23ae442284b0aa8b2403b256)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:31.587655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:31.632732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '391c78ff23ae442284b0aa8b2403b256'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:31.633301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '391c78ff23ae442284b0aa8b2403b256'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:31.640130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '391c78ff23ae442284b0aa8b2403b256'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:31.651660 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '391c78ff23ae442284b0aa8b2403b256'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:31.652366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '391c78ff23ae442284b0aa8b2403b256'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:31.664611 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1012/2062] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:31 2022] DELETE /identity/v3/users/391c78ff23ae442284b0aa8b2403b256 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:31.669674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:31.670182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:31.670352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:31.689424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:31.690598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:31.746933 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1051/2063] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:31.756764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:31.758150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:31.814341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f0382-c2a2-48d5-b7e1-576f74ad0232 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:31.817939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:31.818545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:31.818822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:31.819091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:31.828607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:31.830024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:31.840454 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1013/2064] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:32.103591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:32.105072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:32.121091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c239786-3ab5-442c-80cc-76396996d38d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:32.125710 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:32.126516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:32.126974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:32.127436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] PATH_INFO: `/v3/projects/ceb717ba574f4d65be22d6bf8f4897e8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:32.129650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb717ba574f4d65be22d6bf8f4897e8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:32.133278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:32.179671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb717ba574f4d65be22d6bf8f4897e8', 'request_id': 'req-6f354fa3-35cc-4614-bb75-7014a1be8f69'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.187388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb717ba574f4d65be22d6bf8f4897e8', 'request_id': 'req-6f354fa3-35cc-4614-bb75-7014a1be8f69'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.193714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb717ba574f4d65be22d6bf8f4897e8', 'request_id': 'req-6f354fa3-35cc-4614-bb75-7014a1be8f69'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.204534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] The token cache is being invalidate because project ceb717ba574f4d65be22d6bf8f4897e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:32.204820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.205450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.206919 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1052/2065] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:32 2022] DELETE /identity/v3/projects/ceb717ba574f4d65be22d6bf8f4897e8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:32.210835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:32.220250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84197323-102f-4506-9e02-a6b842e6fb5c service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:32.259653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:32.277331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:32.277970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:32.278247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:32.278520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] PATH_INFO: `/v3/users/ae9877df15354fb69e7282148b89bab0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:32.279633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9877df15354fb69e7282148b89bab0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:32.281568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:32.328845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9877df15354fb69e7282148b89bab0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.329596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9877df15354fb69e7282148b89bab0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.336801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9877df15354fb69e7282148b89bab0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.348876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9877df15354fb69e7282148b89bab0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.349510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-42a29c0b-ddb0-45f8-b2b8-404a378c1174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9877df15354fb69e7282148b89bab0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.362552 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1014/2066] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:32 2022] DELETE /identity/v3/users/ae9877df15354fb69e7282148b89bab0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:32.634093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:32.635531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:32.651326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f354fa3-35cc-4614-bb75-7014a1be8f69 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:32.706792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:32.707446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:32.707671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:32.707939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] PATH_INFO: `/v3/projects/39b94e63060d4a2a84d658f6effb1345` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:32.709055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b94e63060d4a2a84d658f6effb1345)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:32.710991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:32.719124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:32.757878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b94e63060d4a2a84d658f6effb1345', 'request_id': 'req-fccba15f-e403-4e6a-8eae-24f63f1c1172'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.764656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b94e63060d4a2a84d658f6effb1345', 'request_id': 'req-fccba15f-e403-4e6a-8eae-24f63f1c1172'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.770796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b94e63060d4a2a84d658f6effb1345', 'request_id': 'req-fccba15f-e403-4e6a-8eae-24f63f1c1172'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.781600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] The token cache is being invalidate because project 39b94e63060d4a2a84d658f6effb1345 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:32.781864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.782479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fccba15f-e403-4e6a-8eae-24f63f1c1172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:32.783937 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1053/2067] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:32 2022] DELETE /identity/v3/projects/39b94e63060d4a2a84d658f6effb1345 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:32.788948 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079775dc-5b63-44ce-84ba-2e638a6dc510 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:32.789423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079775dc-5b63-44ce-84ba-2e638a6dc510 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:32.789527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079775dc-5b63-44ce-84ba-2e638a6dc510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:32.808996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-079775dc-5b63-44ce-84ba-2e638a6dc510 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:32.810184 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-079775dc-5b63-44ce-84ba-2e638a6dc510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:32.868711 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1015/2068] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:32.877596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168406c6-f23e-4bc6-b99f-433d1002b952 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:32.878145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168406c6-f23e-4bc6-b99f-433d1002b952 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:32.878423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168406c6-f23e-4bc6-b99f-433d1002b952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:32.898011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-168406c6-f23e-4bc6-b99f-433d1002b952 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:32.899206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-168406c6-f23e-4bc6-b99f-433d1002b952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:32.907767 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1054/2069] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:32.911912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079775dc-5b63-44ce-84ba-2e638a6dc510 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:32.921973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079775dc-5b63-44ce-84ba-2e638a6dc510 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:32.926461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:32.927051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:32.927327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:32.927597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:32.928230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:32.928558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:32.930183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:32.941537 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1016/2070] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:02:32.945429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168406c6-f23e-4bc6-b99f-433d1002b952 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:32.954462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168406c6-f23e-4bc6-b99f-433d1002b952 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:32.959060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:32.959647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:32.959930 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:32.960202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:32.960863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:32.962839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:32.990154 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1055/2071] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:32 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:32.994088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.003929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c1b3d2-b8b2-44d4-aea1-b923b47f62c8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.046975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:33.059662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.060256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.060559 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.060827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.061453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.063326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.083691 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1017/2072] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:32 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:33.087807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.098402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5421ec-00a4-4963-b2d9-d73d37726195 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.102939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.103520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.103794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.104062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.104753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.106320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.115833 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1056/2073] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:33.119470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.129338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881a770-ed39-4364-8b03-55cd1dd73f00 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.133820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d4cd6-9f95-4c36-b25c-5e15c26ce53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.134399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d4cd6-9f95-4c36-b25c-5e15c26ce53b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.134675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d4cd6-9f95-4c36-b25c-5e15c26ce53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.134942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d4cd6-9f95-4c36-b25c-5e15c26ce53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.135571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d4cd6-9f95-4c36-b25c-5e15c26ce53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.137688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d4cd6-9f95-4c36-b25c-5e15c26ce53b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.147957 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1018/2074] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:33.152478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.162581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81667e9-b3c9-451f-929d-3550edd34423 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.167343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.167995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.168273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.168591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] PATH_INFO: `/v3/projects/dae5cc43bae445389aa1b05c89040a2d/users/7cb6e613f5c74c68b7e12bca18e6b9f2/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.183037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dae5cc43bae445389aa1b05c89040a2d, user_id=7cb6e613f5c74c68b7e12bca18e6b9f2, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.185785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.198193 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1057/2075] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:33 2022] PUT /identity/v3/projects/dae5cc43bae445389aa1b05c89040a2d/users/7cb6e613f5c74c68b7e12bca18e6b9f2/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:02:33.208829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f38399-a911-41fb-bea6-d0536cd87889 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.209114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f38399-a911-41fb-bea6-d0536cd87889 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.209392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f38399-a911-41fb-bea6-d0536cd87889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.236764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a6f38399-a911-41fb-bea6-d0536cd87889 None None] MFA Rules not processed for user `7cb6e613f5c74c68b7e12bca18e6b9f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:33.237883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a6f38399-a911-41fb-bea6-d0536cd87889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:33.300920 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1019/2076] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:33.305371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.364845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d496321-f418-4bb3-89d7-70fb5c6f062b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.369462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.370063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.370344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.370617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.371755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.373684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.399959 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1058/2077] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:33 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:33.404164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f38399-a911-41fb-bea6-d0536cd87889 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.415609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f38399-a911-41fb-bea6-d0536cd87889 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.471033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.471644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.471926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.472201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.472873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.474943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.495655 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1020/2078] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:33 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:33.500188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.510529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114b30c-e2dc-4e24-9a62-746f21db9ae4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.515695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.516279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.516587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.516856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.517484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.519055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.529614 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1059/2079] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:33.533513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.543125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dea976-9f11-4840-afc4-564705967418 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.547637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.548274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.548578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.548848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] PATH_INFO: `/v3/projects/ad8bb3fbd3ec4810aa6e5f4620c137ab/users/5603db11b24745fc92211935e36b87ed/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.564684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8bb3fbd3ec4810aa6e5f4620c137ab, user_id=5603db11b24745fc92211935e36b87ed, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.567300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.581033 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1021/2080] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:33 2022] PUT /identity/v3/projects/ad8bb3fbd3ec4810aa6e5f4620c137ab/users/5603db11b24745fc92211935e36b87ed/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:33.584981 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.595794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955d703-7f69-4d2b-a610-aa964fbbc5bd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.599297 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:33.653162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b004a84c-3bcd-45de-b769-a93456d87354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.653760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004a84c-3bcd-45de-b769-a93456d87354 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.654039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004a84c-3bcd-45de-b769-a93456d87354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.654326 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b004a84c-3bcd-45de-b769-a93456d87354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.654944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004a84c-3bcd-45de-b769-a93456d87354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.656547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b004a84c-3bcd-45de-b769-a93456d87354 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.666207 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1060/2081] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:33.670246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.680369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1aea4c-907f-4774-bd1b-610f75442b15 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.685407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.686049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.686322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.686586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] PATH_INFO: `/v3/projects/ad8bb3fbd3ec4810aa6e5f4620c137ab/users/5603db11b24745fc92211935e36b87ed/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.688500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8bb3fbd3ec4810aa6e5f4620c137ab, user_id=5603db11b24745fc92211935e36b87ed, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.691087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.701467 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1022/2082] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:33 2022] PUT /identity/v3/projects/ad8bb3fbd3ec4810aa6e5f4620c137ab/users/5603db11b24745fc92211935e36b87ed/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:33.711456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4a86-8dfc-467e-8b0d-8d6f6450cea5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.711956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4a86-8dfc-467e-8b0d-8d6f6450cea5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.712064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d4a86-8dfc-467e-8b0d-8d6f6450cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.740601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-057d4a86-8dfc-467e-8b0d-8d6f6450cea5 None None] MFA Rules not processed for user `5603db11b24745fc92211935e36b87ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:33.741802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-057d4a86-8dfc-467e-8b0d-8d6f6450cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:33.807855 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1061/2083] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:33 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:33.821057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:33.822798 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:33.833578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:33.880919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc83494f-143c-45c0-8dcf-04146cadb0db admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:33.884579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:33.885181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:33.885459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:33.885729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:33.894575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:33.895978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:33.905242 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1023/2084] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:33 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:34.209692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4a86-8dfc-467e-8b0d-8d6f6450cea5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:34.218905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d4a86-8dfc-467e-8b0d-8d6f6450cea5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:34.223483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5603db11b24745fc92211935e36b87ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad8bb3fbd3ec4810aa6e5f4620c137ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:34.224084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:34.224387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:34.224660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:34.225300 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:34.225597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:34.227203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:34.238345 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1062/2085] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:02:34.242474 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:34.251886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f8c50-04bd-4bef-98a6-c220e2c4ebb1 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:34.256723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5603db11b24745fc92211935e36b87ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad8bb3fbd3ec4810aa6e5f4620c137ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:34.257323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:34.257604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:34.257872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:34.258502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:34.260382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:34.287045 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1024/2086] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:34 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:34.291207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:34.301463 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec9dca6-0192-49f6-8b2d-a30461a2a33a tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:34.356145 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5603db11b24745fc92211935e36b87ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad8bb3fbd3ec4810aa6e5f4620c137ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:34.356703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:34.356985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:34.357260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:34.357897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:34.358199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:34.359792 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:34.371800 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1063/2087] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:02:34.408578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:34.418541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8219f-b1d4-4de8-bf44-4de8686b19ce tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:34.423223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5603db11b24745fc92211935e36b87ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ad8bb3fbd3ec4810aa6e5f4620c137ab', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:34.423850 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:34.424158 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:34.424457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] PATH_INFO: `/v3/projects/6822784c4b2341108c02830eab3bdb9f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:34.425613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6822784c4b2341108c02830eab3bdb9f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:34.427489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:34.473795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6822784c4b2341108c02830eab3bdb9f', 'request_id': 'req-1e7101b2-04e8-436c-962d-063335f99476'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.481751 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6822784c4b2341108c02830eab3bdb9f', 'request_id': 'req-1e7101b2-04e8-436c-962d-063335f99476'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.487849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6822784c4b2341108c02830eab3bdb9f', 'request_id': 'req-1e7101b2-04e8-436c-962d-063335f99476'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.499712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] The token cache is being invalidate because project 6822784c4b2341108c02830eab3bdb9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:34.499996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.500654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1e7101b2-04e8-436c-962d-063335f99476 tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.502134 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1025/2088] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:34 2022] DELETE /identity/v3/projects/6822784c4b2341108c02830eab3bdb9f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:34.691195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:34.692925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:34.703513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:34.756641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c6ccf-a6d5-4644-874b-3dbf2559ab8c tempest-VolumeQuotaClassesTest-1390113830 tempest-VolumeQuotaClassesTest-1390113830-project-admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:34.810720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:34.811356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:34.811638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:34.811915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] PATH_INFO: `/v3/users/7cb6e613f5c74c68b7e12bca18e6b9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:34.813073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cb6e613f5c74c68b7e12bca18e6b9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:34.815007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:34.859898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cb6e613f5c74c68b7e12bca18e6b9f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.866817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cb6e613f5c74c68b7e12bca18e6b9f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.873944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cb6e613f5c74c68b7e12bca18e6b9f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.885853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cb6e613f5c74c68b7e12bca18e6b9f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.886490 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cb6e613f5c74c68b7e12bca18e6b9f2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:34.892246 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1064/2089] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:34 2022] DELETE /identity/v3/users/7cb6e613f5c74c68b7e12bca18e6b9f2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:34.896706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:34.897181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:34.897294 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:34.915941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:34.917155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:34.974013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1026/2090] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:34.984191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:34.985653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:35.024460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:35.044093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09eb8c8-7807-460e-b124-9fa228e4b30f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:35.047731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:35.048402 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:35.048675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:35.048943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:35.057603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:35.059133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:35.068236 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1065/2091] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:35.338630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:35.341086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:35.353767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68f0cb-48b0-41e7-a93a-a3e58d21927e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:35.358801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:35.359467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:35.359748 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:35.360024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] PATH_INFO: `/v3/projects/dae5cc43bae445389aa1b05c89040a2d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:35.361464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dae5cc43bae445389aa1b05c89040a2d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:35.363648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:35.373678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:35.419859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dae5cc43bae445389aa1b05c89040a2d', 'request_id': 'req-01f03842-0847-490c-935c-3f809546a188'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.429776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dae5cc43bae445389aa1b05c89040a2d', 'request_id': 'req-01f03842-0847-490c-935c-3f809546a188'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.436225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dae5cc43bae445389aa1b05c89040a2d', 'request_id': 'req-01f03842-0847-490c-935c-3f809546a188'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.447483 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] The token cache is being invalidate because project dae5cc43bae445389aa1b05c89040a2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:35.447771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.448430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.450431 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1027/2092] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:35 2022] DELETE /identity/v3/projects/dae5cc43bae445389aa1b05c89040a2d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:35.454324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:35.463901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471d496-d2ee-4e9d-bcb4-95c2678af9cd service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:35.518976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:35.519600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:35.519877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:35.520151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] PATH_INFO: `/v3/users/5603db11b24745fc92211935e36b87ed` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:35.521296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5603db11b24745fc92211935e36b87ed)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:35.523221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:35.567513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5603db11b24745fc92211935e36b87ed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.574745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5603db11b24745fc92211935e36b87ed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.581989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5603db11b24745fc92211935e36b87ed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.594741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5603db11b24745fc92211935e36b87ed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.595368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-e2924624-dd40-4b80-a91c-0f38319dc630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5603db11b24745fc92211935e36b87ed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.601135 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1066/2093] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:35 2022] DELETE /identity/v3/users/5603db11b24745fc92211935e36b87ed => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:35.864272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:35.865989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:35.879287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f03842-0847-490c-935c-3f809546a188 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:35.934605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:35.935235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:35.935511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:35.935784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] PATH_INFO: `/v3/projects/ad8bb3fbd3ec4810aa6e5f4620c137ab` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:35.936902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8bb3fbd3ec4810aa6e5f4620c137ab)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:35.938800 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:35.984862 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8bb3fbd3ec4810aa6e5f4620c137ab', 'request_id': 'req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.991678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8bb3fbd3ec4810aa6e5f4620c137ab', 'request_id': 'req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:35.997240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8bb3fbd3ec4810aa6e5f4620c137ab', 'request_id': 'req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:36.008611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] The token cache is being invalidate because project ad8bb3fbd3ec4810aa6e5f4620c137ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:36.008895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:36.009516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa5b1b6a-936f-4539-a8cd-f08acd23e8b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:36.010981 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1028/2094] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:35 2022] DELETE /identity/v3/projects/ad8bb3fbd3ec4810aa6e5f4620c137ab => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:36.016471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071fed7-303a-4e4c-9fe8-a976f54b0098 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.016952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071fed7-303a-4e4c-9fe8-a976f54b0098 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.017072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071fed7-303a-4e4c-9fe8-a976f54b0098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.036807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7071fed7-303a-4e4c-9fe8-a976f54b0098 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:36.038000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7071fed7-303a-4e4c-9fe8-a976f54b0098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:36.086592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:36.098583 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1067/2095] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:36.107513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8af36-a07b-425b-b353-761c2c5d6c86 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.107804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8af36-a07b-425b-b353-761c2c5d6c86 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.108077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd8af36-a07b-425b-b353-761c2c5d6c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.128081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-ebd8af36-a07b-425b-b353-761c2c5d6c86 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:36.129328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ebd8af36-a07b-425b-b353-761c2c5d6c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:36.137120 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1029/2096] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:36.141260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071fed7-303a-4e4c-9fe8-a976f54b0098 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.154863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071fed7-303a-4e4c-9fe8-a976f54b0098 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.159381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.159968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.160241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.160534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.161168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:36.161466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.163032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.174997 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1068/2097] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:02:36.178970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8af36-a07b-425b-b353-761c2c5d6c86 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.188698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd8af36-a07b-425b-b353-761c2c5d6c86 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.193272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.193868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.194148 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.194419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.195051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.196996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.219330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:36.227708 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1030/2098] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:36.232379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.242244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f1204-1d86-4838-80fe-b83fd218cc05 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.298404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.299000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.299275 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.299543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.300174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.302117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.322600 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1069/2099] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:36.326643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.336546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e2954-62a7-4517-883f-13d49d1a60f1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.341525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.342110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.342409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.342678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.343314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.344905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.354593 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1031/2100] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:36.358818 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.368470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446d507-7130-4f3b-b7f1-759482638336 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.373269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b6cc8-c725-4a18-a4b5-11a9452aa9a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.373864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b6cc8-c725-4a18-a4b5-11a9452aa9a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.374139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b6cc8-c725-4a18-a4b5-11a9452aa9a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.374440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b6cc8-c725-4a18-a4b5-11a9452aa9a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.375091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b6cc8-c725-4a18-a4b5-11a9452aa9a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.376697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b6cc8-c725-4a18-a4b5-11a9452aa9a1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.387349 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1070/2101] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:36.391250 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.400919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41a693-ff21-4998-9de9-e584a8714693 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.405916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.406563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.406822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.407089 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] PATH_INFO: `/v3/projects/5acc51799adc4c7b9a627940b4a71867/users/4208f810eea449608f0ed197232690d0/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.421992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5acc51799adc4c7b9a627940b4a71867, user_id=4208f810eea449608f0ed197232690d0, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.424707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.437167 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1032/2102] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:36 2022] PUT /identity/v3/projects/5acc51799adc4c7b9a627940b4a71867/users/4208f810eea449608f0ed197232690d0/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:02:36.447357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7d820-48b7-4594-96d8-e9778b611a78 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.447639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7d820-48b7-4594-96d8-e9778b611a78 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.447911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7d820-48b7-4594-96d8-e9778b611a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.475762 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e6b7d820-48b7-4594-96d8-e9778b611a78 None None] MFA Rules not processed for user `4208f810eea449608f0ed197232690d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:36.477014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e6b7d820-48b7-4594-96d8-e9778b611a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:36.539404 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1071/2103] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:36.544539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.604514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db5d6b-6702-47af-aa69-acba2d8b050c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.609726 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.610320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.610596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.610867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.611487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.613403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.640268 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1033/2104] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:36.644354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7d820-48b7-4594-96d8-e9778b611a78 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.654838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7d820-48b7-4594-96d8-e9778b611a78 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.701090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:36.711117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.711717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.711995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.712263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.712914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.714844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.735656 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1072/2105] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:36.739588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.750519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ec200c-0f52-4e72-b587-b6988578be77 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.755331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.755926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.756204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.756509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.757145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.758715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.769461 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1034/2106] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:36.773602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.783457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad545ec-b31e-4201-a2a5-ae80c0610083 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.787973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.788649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.788928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.789201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] PATH_INFO: `/v3/projects/d538b58a87a546109afca8bf5b8c7f0a/users/ef82a9799f7943b5b7800c5d177f77db/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.803886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d538b58a87a546109afca8bf5b8c7f0a, user_id=ef82a9799f7943b5b7800c5d177f77db, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.806552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.819489 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1073/2107] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:36 2022] PUT /identity/v3/projects/d538b58a87a546109afca8bf5b8c7f0a/users/ef82a9799f7943b5b7800c5d177f77db/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:36.823549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.833950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94454de3-37d1-473c-aeb4-4fa528caa4d6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.889703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae2ffc-6229-4984-ad21-da4e5f10d301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.890216 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae2ffc-6229-4984-ad21-da4e5f10d301 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.890493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae2ffc-6229-4984-ad21-da4e5f10d301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.890762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae2ffc-6229-4984-ad21-da4e5f10d301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.891386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae2ffc-6229-4984-ad21-da4e5f10d301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.892988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae2ffc-6229-4984-ad21-da4e5f10d301 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.903037 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1035/2108] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:36.907180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:36.917451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544243b7-a3e9-4126-a7c4-fa4e471e13cb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:36.922062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:36.922706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.922986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.923261 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] PATH_INFO: `/v3/projects/d538b58a87a546109afca8bf5b8c7f0a/users/ef82a9799f7943b5b7800c5d177f77db/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.925181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d538b58a87a546109afca8bf5b8c7f0a, user_id=ef82a9799f7943b5b7800c5d177f77db, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:36.927815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:36.937847 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1074/2109] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:36 2022] PUT /identity/v3/projects/d538b58a87a546109afca8bf5b8c7f0a/users/ef82a9799f7943b5b7800c5d177f77db/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:36.948581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859d814-d002-4220-b4f7-716dfc0e6d51 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:36.949536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859d814-d002-4220-b4f7-716dfc0e6d51 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:36.949656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859d814-d002-4220-b4f7-716dfc0e6d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:36.977571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7859d814-d002-4220-b4f7-716dfc0e6d51 None None] MFA Rules not processed for user `ef82a9799f7943b5b7800c5d177f77db`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:36.978767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7859d814-d002-4220-b4f7-716dfc0e6d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:36.998114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:37.046653 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1036/2110] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:37.057708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:37.059366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:37.115064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15948fb5-99c4-419e-846b-1e47995cf3a2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:37.118695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:37.119282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:37.119561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:37.119834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:37.129472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:37.130890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:37.140718 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1075/2111] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:37.221395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859d814-d002-4220-b4f7-716dfc0e6d51 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:37.230985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7859d814-d002-4220-b4f7-716dfc0e6d51 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:37.234542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8237d1-3823-40d5-80e3-91b5ab7bc1bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:37.235136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8237d1-3823-40d5-80e3-91b5ab7bc1bb service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:37.235410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8237d1-3823-40d5-80e3-91b5ab7bc1bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:37.235678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8237d1-3823-40d5-80e3-91b5ab7bc1bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:37.291383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8237d1-3823-40d5-80e3-91b5ab7bc1bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:37.292859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8237d1-3823-40d5-80e3-91b5ab7bc1bb service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:37.301538 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1037/2112] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:42.083752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:42.145857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a8ef41-8e00-47f7-a8a1-696475bb01c1 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:42.152684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:42.153306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:42.153580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:42.153847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] PATH_INFO: `/v3/users/4208f810eea449608f0ed197232690d0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:42.154958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4208f810eea449608f0ed197232690d0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:42.156889 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:42.204820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4208f810eea449608f0ed197232690d0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.206769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:42.213871 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4208f810eea449608f0ed197232690d0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.221356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4208f810eea449608f0ed197232690d0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.233422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4208f810eea449608f0ed197232690d0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.234050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4208f810eea449608f0ed197232690d0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.239764 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1076/2113] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:42 2022] DELETE /identity/v3/users/4208f810eea449608f0ed197232690d0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:42.244429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:42.244903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:42.245001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:42.264269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:42.265429 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:42.323552 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1038/2114] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:42.332945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:42.334400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:42.390728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b825759-80f5-4980-9510-fd316e2b624c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:42.394408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:42.395012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:42.395293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:42.395564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:42.404999 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:42.406431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:42.416049 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1077/2115] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:42.680313 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:42.681813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:42.693595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6dc107-d848-4a8d-a791-e9daa56ca5fc None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:42.698102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:42.698734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:42.699039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:42.699311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] PATH_INFO: `/v3/projects/5acc51799adc4c7b9a627940b4a71867` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:42.700459 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] RBAC: Authorizing `identity:delete_project(project_id=5acc51799adc4c7b9a627940b4a71867)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:42.702347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:42.748126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5acc51799adc4c7b9a627940b4a71867', 'request_id': 'req-1c36899f-4a33-479b-a949-b62eb7bbe0de'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.750588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:42.755661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5acc51799adc4c7b9a627940b4a71867', 'request_id': 'req-1c36899f-4a33-479b-a949-b62eb7bbe0de'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.762878 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5acc51799adc4c7b9a627940b4a71867', 'request_id': 'req-1c36899f-4a33-479b-a949-b62eb7bbe0de'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.773645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] The token cache is being invalidate because project 5acc51799adc4c7b9a627940b4a71867 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:42.773865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.774496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.775955 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1039/2116] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:42 2022] DELETE /identity/v3/projects/5acc51799adc4c7b9a627940b4a71867 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:42.780711 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:42.790084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5b111-74f4-4865-8ba2-98a6108a2fe9 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:42.848131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:42.848813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:42.849090 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:42.849362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] PATH_INFO: `/v3/users/ef82a9799f7943b5b7800c5d177f77db` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:42.850503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef82a9799f7943b5b7800c5d177f77db)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:42.852494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:42.898900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef82a9799f7943b5b7800c5d177f77db'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.906767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef82a9799f7943b5b7800c5d177f77db'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.914794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef82a9799f7943b5b7800c5d177f77db'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.923159 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:42.928955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef82a9799f7943b5b7800c5d177f77db'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.929945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8af4e5b1-9b09-45c2-8a76-ca681660ff75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef82a9799f7943b5b7800c5d177f77db'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:42.937092 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1078/2117] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:42 2022] DELETE /identity/v3/users/ef82a9799f7943b5b7800c5d177f77db => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:43.217886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.219353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:43.235654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c36899f-4a33-479b-a949-b62eb7bbe0de admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.314934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.315625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.316345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.316345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] PATH_INFO: `/v3/projects/d538b58a87a546109afca8bf5b8c7f0a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.317379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d538b58a87a546109afca8bf5b8c7f0a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.319396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:43.369693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd538b58a87a546109afca8bf5b8c7f0a', 'request_id': 'req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:43.377129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd538b58a87a546109afca8bf5b8c7f0a', 'request_id': 'req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:43.383129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd538b58a87a546109afca8bf5b8c7f0a', 'request_id': 'req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:43.394002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] The token cache is being invalidate because project d538b58a87a546109afca8bf5b8c7f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:43.394283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:43.394903 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0fd1c148-554f-4f02-9a75-eaf7fcfc6034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:43.396407 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1040/2118] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:43 2022] DELETE /identity/v3/projects/d538b58a87a546109afca8bf5b8c7f0a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:43.401282 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0f05-5307-4154-9e4a-bea96ff542ae None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.401746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0f05-5307-4154-9e4a-bea96ff542ae None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.402391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899b0f05-5307-4154-9e4a-bea96ff542ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.421812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-899b0f05-5307-4154-9e4a-bea96ff542ae None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:43.422990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-899b0f05-5307-4154-9e4a-bea96ff542ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:43.479262 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1079/2119] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:43.487356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05f039-1e93-4f81-9e50-90fb2665ecc1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.487664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05f039-1e93-4f81-9e50-90fb2665ecc1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.487937 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd05f039-1e93-4f81-9e50-90fb2665ecc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.508760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bd05f039-1e93-4f81-9e50-90fb2665ecc1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:43.509939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bd05f039-1e93-4f81-9e50-90fb2665ecc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:43.518125 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1041/2120] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:43.522237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0f05-5307-4154-9e4a-bea96ff542ae None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.532585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899b0f05-5307-4154-9e4a-bea96ff542ae None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.537134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.537736 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.538018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.538290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.538929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:43.539233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.540858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:43.552737 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1080/2121] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:02:43.556732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05f039-1e93-4f81-9e50-90fb2665ecc1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.565723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd05f039-1e93-4f81-9e50-90fb2665ecc1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.570196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.570779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.571052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.571317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.571941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.574370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:43.604518 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1042/2122] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:43 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:43.605692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.615419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0eb243-94ab-4d36-b00c-f7ed4ccae7fb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.670717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.671328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.671631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.671896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.672555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.674457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:43.694873 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1081/2123] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:43 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:43.699423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.709642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f07075-aafa-4aec-aca7-944eb78160e6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.715311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.715959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.716703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.716703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.717168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.718732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:43.728891 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1043/2124] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:43.732915 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.742617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31fa753-f9c3-4ccb-b4e8-39c5824f25ae admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.747193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c0e7b-e5cd-4bd8-a0c1-211f5f6ce8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.747777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c0e7b-e5cd-4bd8-a0c1-211f5f6ce8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.748054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c0e7b-e5cd-4bd8-a0c1-211f5f6ce8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.748353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c0e7b-e5cd-4bd8-a0c1-211f5f6ce8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.748990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c0e7b-e5cd-4bd8-a0c1-211f5f6ce8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.750606 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c0e7b-e5cd-4bd8-a0c1-211f5f6ce8f6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:43.761167 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1082/2125] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:43.765792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.775663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769e46f-8c15-4410-a9b7-069498c8c2c0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.781168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.781827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.782104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.782379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] PATH_INFO: `/v3/projects/dd420ddd7a9a46debee3779334689163/users/48f3b1272d6e45029160da91162a34ff/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.797022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd420ddd7a9a46debee3779334689163, user_id=48f3b1272d6e45029160da91162a34ff, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.799581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:43.811786 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1044/2126] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:43 2022] PUT /identity/v3/projects/dd420ddd7a9a46debee3779334689163/users/48f3b1272d6e45029160da91162a34ff/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:43.821911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94204eb-9130-4bf9-82e3-887f2debd1db None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.822191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94204eb-9130-4bf9-82e3-887f2debd1db None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.822488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94204eb-9130-4bf9-82e3-887f2debd1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.850337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-a94204eb-9130-4bf9-82e3-887f2debd1db None None] MFA Rules not processed for user `48f3b1272d6e45029160da91162a34ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:43.851521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a94204eb-9130-4bf9-82e3-887f2debd1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:43.894296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:43.917707 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1083/2127] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:43.922304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:43.931079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:43.986249 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d97859-dbbf-4875-a26d-3313ed1be9db admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:43.990911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:43.991506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:43.991780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:43.992047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:43.992712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:43.994629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:44.024174 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1045/2128] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:43 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:44.028132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94204eb-9130-4bf9-82e3-887f2debd1db None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:44.039406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94204eb-9130-4bf9-82e3-887f2debd1db None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:44.103868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:44.104513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:44.104788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:44.105106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:44.105748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:44.107687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:44.130693 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1084/2129] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:44 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:02:44.135127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:44.147994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d5bb84-c436-4717-864b-844296b95732 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:44.152803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:44.153412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:44.153685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:44.153975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:44.154602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:44.156152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:44.166981 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1046/2130] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:44.171141 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:44.181723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe77f3-5b47-4b6f-8e79-c5cf6a943b69 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:44.186920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:44.187571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:44.187853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:44.188129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] PATH_INFO: `/v3/projects/9769d77b21204dcab47dc83b10ac6a45/users/2b4c53508669459398315e3c15af6701/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:44.203328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9769d77b21204dcab47dc83b10ac6a45, user_id=2b4c53508669459398315e3c15af6701, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:44.206024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:44.218208 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1085/2131] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:44 2022] PUT /identity/v3/projects/9769d77b21204dcab47dc83b10ac6a45/users/2b4c53508669459398315e3c15af6701/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:44.222146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:44.232745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341602b5-26c5-467a-8e0a-b5a478f09dff admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:44.287120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecbdc2f-2283-43b8-8ac8-88b3e0ab9a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:44.287720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbdc2f-2283-43b8-8ac8-88b3e0ab9a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:44.288000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbdc2f-2283-43b8-8ac8-88b3e0ab9a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:44.288271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecbdc2f-2283-43b8-8ac8-88b3e0ab9a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:44.288970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbdc2f-2283-43b8-8ac8-88b3e0ab9a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:44.290521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecbdc2f-2283-43b8-8ac8-88b3e0ab9a25 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:44.300663 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1047/2132] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:44.304717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:44.315346 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ee47a-35e2-4478-8a45-5ef75528f3fd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:44.319923 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:44.320593 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:44.320877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:44.321149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] PATH_INFO: `/v3/projects/9769d77b21204dcab47dc83b10ac6a45/users/2b4c53508669459398315e3c15af6701/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:44.323030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9769d77b21204dcab47dc83b10ac6a45, user_id=2b4c53508669459398315e3c15af6701, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:44.326590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:44.336759 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1086/2133] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:44 2022] PUT /identity/v3/projects/9769d77b21204dcab47dc83b10ac6a45/users/2b4c53508669459398315e3c15af6701/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:44.348526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:44.348944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:44.349041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:44.377662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] MFA Rules not processed for user `2b4c53508669459398315e3c15af6701`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:44.379122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:44.451837 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1048/2134] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:02:44.461924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:44.463729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:44.521789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b0963d-e679-4e53-a5f6-5cdc775b5edf admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:44.525480 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ebab23-c0ff-42cb-84cc-0e49ee4ab3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:44.526085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebab23-c0ff-42cb-84cc-0e49ee4ab3de service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:44.526358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebab23-c0ff-42cb-84cc-0e49ee4ab3de service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:44.526624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ebab23-c0ff-42cb-84cc-0e49ee4ab3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:44.581737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:44.593605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebab23-c0ff-42cb-84cc-0e49ee4ab3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:44.595021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ebab23-c0ff-42cb-84cc-0e49ee4ab3de service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:44.604140 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1087/2135] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:48.876387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:48.877773 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:48.885369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:48.892103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988d347-ff0c-452d-a3f1-2975b2c11a23 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:48.895979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:48.896622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:48.896902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:48.897192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:48.907253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:48.908984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:48.919172 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1049/2136] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:50.084013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2f6d-a230-44a7-8f51-dc6342e616dd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:50.084589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2f6d-a230-44a7-8f51-dc6342e616dd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:50.084589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b2f6d-a230-44a7-8f51-dc6342e616dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:50.105856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-d21b2f6d-a230-44a7-8f51-dc6342e616dd None None] MFA Rules not processed for user `959991a17c69459dbd665c2ae20095ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:50.107066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d21b2f6d-a230-44a7-8f51-dc6342e616dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:50.161812 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1088/2137] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:53.867179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:53.931441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2c117-96f1-4081-b4b6-2897dfaac3ff service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:53.935974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:53.936954 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:53.937240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:53.937513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] PATH_INFO: `/v3/users/48f3b1272d6e45029160da91162a34ff` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:53.939210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48f3b1272d6e45029160da91162a34ff)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:53.941152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:53.993006 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48f3b1272d6e45029160da91162a34ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:53.993775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48f3b1272d6e45029160da91162a34ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.000707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48f3b1272d6e45029160da91162a34ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.013486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48f3b1272d6e45029160da91162a34ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.014126 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48f3b1272d6e45029160da91162a34ff'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.026433 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1050/2138] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:53 2022] DELETE /identity/v3/users/48f3b1272d6e45029160da91162a34ff => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:54.031379 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:54.031877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:54.031977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:54.050336 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:54.051499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:54.107666 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1089/2139] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:54.117345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:54.118720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:54.176417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e494cc2-d737-498f-9c73-f69b6cba92f3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:54.179898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:54.180530 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:54.180804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:54.181073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:54.190025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:54.191436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:54.201524 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1051/2140] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:02:54.532369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:54.533776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:54.545319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7534599d-3f9a-4155-981a-754a430fc36c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:54.549886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:54.550505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:54.550783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:54.551058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] PATH_INFO: `/v3/projects/dd420ddd7a9a46debee3779334689163` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:54.552166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd420ddd7a9a46debee3779334689163)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:54.554098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:54.601815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd420ddd7a9a46debee3779334689163', 'request_id': 'req-d95820f6-d38d-4fa1-88d7-42f8e85229b2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.608243 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd420ddd7a9a46debee3779334689163', 'request_id': 'req-d95820f6-d38d-4fa1-88d7-42f8e85229b2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.614352 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd420ddd7a9a46debee3779334689163', 'request_id': 'req-d95820f6-d38d-4fa1-88d7-42f8e85229b2'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.624846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] The token cache is being invalidate because project dd420ddd7a9a46debee3779334689163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:54.625121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.625729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.627213 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1090/2141] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:54 2022] DELETE /identity/v3/projects/dd420ddd7a9a46debee3779334689163 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:54.631379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:54.641361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13120ed9-a547-4cbc-83b8-7598eec12050 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:54.648909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:54.698238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:54.699462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:54.699739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:54.700010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] PATH_INFO: `/v3/users/2b4c53508669459398315e3c15af6701` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:54.701278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b4c53508669459398315e3c15af6701)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:54.703267 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:54.748067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b4c53508669459398315e3c15af6701'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.748729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b4c53508669459398315e3c15af6701'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.755830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b4c53508669459398315e3c15af6701'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.767463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b4c53508669459398315e3c15af6701'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.768114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-50e14086-98b3-4638-bb12-e3e0aa6f8490 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b4c53508669459398315e3c15af6701'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:54.779967 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1052/2142] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:02:54 2022] DELETE /identity/v3/users/2b4c53508669459398315e3c15af6701 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:55.046496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.048231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:55.067134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95820f6-d38d-4fa1-88d7-42f8e85229b2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.094492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:55.138060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.138692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.138972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.139240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] PATH_INFO: `/v3/projects/9769d77b21204dcab47dc83b10ac6a45` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.140404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9769d77b21204dcab47dc83b10ac6a45)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.142290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.197118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9769d77b21204dcab47dc83b10ac6a45', 'request_id': 'req-7241a3cf-6480-48e1-8cd4-09bcc566f93b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:55.204907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9769d77b21204dcab47dc83b10ac6a45', 'request_id': 'req-7241a3cf-6480-48e1-8cd4-09bcc566f93b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:55.211202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9769d77b21204dcab47dc83b10ac6a45', 'request_id': 'req-7241a3cf-6480-48e1-8cd4-09bcc566f93b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:55.224064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] The token cache is being invalidate because project 9769d77b21204dcab47dc83b10ac6a45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:02:55.224391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:55.225089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7241a3cf-6480-48e1-8cd4-09bcc566f93b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:02:55.227102 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1091/2143] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:02:55 2022] DELETE /identity/v3/projects/9769d77b21204dcab47dc83b10ac6a45 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:55.233410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432260-b22b-431f-8de7-a9ab5ac3633d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.234214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432260-b22b-431f-8de7-a9ab5ac3633d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.234667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b432260-b22b-431f-8de7-a9ab5ac3633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.255036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3b432260-b22b-431f-8de7-a9ab5ac3633d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:55.256395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3b432260-b22b-431f-8de7-a9ab5ac3633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:55.316739 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1053/2144] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:55.324936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161f43c-7853-4a51-87a6-3fbd899936b2 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.325501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161f43c-7853-4a51-87a6-3fbd899936b2 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.325948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161f43c-7853-4a51-87a6-3fbd899936b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.347027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-1161f43c-7853-4a51-87a6-3fbd899936b2 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:55.348126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-1161f43c-7853-4a51-87a6-3fbd899936b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:55.357048 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1092/2145] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:55.361336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b432260-b22b-431f-8de7-a9ab5ac3633d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.371970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b432260-b22b-431f-8de7-a9ab5ac3633d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.376841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.377455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.377735 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.378270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.378916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:02:55.379217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.380945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.404093 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1054/2146] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:02:55.408658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161f43c-7853-4a51-87a6-3fbd899936b2 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.418908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161f43c-7853-4a51-87a6-3fbd899936b2 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.423868 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.424575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.424855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.425125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.425754 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.427657 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.457378 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1093/2147] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:02:55 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:55.459219 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.470010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f51c29-d48b-4473-b6df-8c5f03611785 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.483584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:55.534975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.535584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.535864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.536143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.536803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.538757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.560469 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1055/2148] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:02:55 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:02:55.564367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.574430 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851afec9-f074-4d2b-808f-c628ed7810a1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.578978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.579568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.579846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.580115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.580828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.583111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.594951 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1094/2149] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:55.599025 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.609448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863b021e-49f9-4413-8667-6a4b35cce95c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.613955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7deaa6-7158-4cda-8b57-1338929f684f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.614541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7deaa6-7158-4cda-8b57-1338929f684f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.614814 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7deaa6-7158-4cda-8b57-1338929f684f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.615082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7deaa6-7158-4cda-8b57-1338929f684f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.615710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7deaa6-7158-4cda-8b57-1338929f684f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.617293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7deaa6-7158-4cda-8b57-1338929f684f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.628465 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1056/2150] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:02:55.632485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.642334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9aec54-10c2-4242-88bc-44a6f32e7bd1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.646959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.647594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.647874 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.648148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] PATH_INFO: `/v3/projects/79cd0f930dca4501b35b200777cee537/users/9748a495a8c044c0a4470a7d3afeecb0/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.662353 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] RBAC: Authorizing `identity:create_grant(project_id=79cd0f930dca4501b35b200777cee537, user_id=9748a495a8c044c0a4470a7d3afeecb0, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.665036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.678178 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1095/2151] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:02:55 2022] PUT /identity/v3/projects/79cd0f930dca4501b35b200777cee537/users/9748a495a8c044c0a4470a7d3afeecb0/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:02:55.691147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055499e-1847-4e3c-ae1c-c80e007a0b77 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.691427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055499e-1847-4e3c-ae1c-c80e007a0b77 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.691694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3055499e-1847-4e3c-ae1c-c80e007a0b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.719095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3055499e-1847-4e3c-ae1c-c80e007a0b77 None None] MFA Rules not processed for user `9748a495a8c044c0a4470a7d3afeecb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:02:55.720290 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3055499e-1847-4e3c-ae1c-c80e007a0b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:55.782633 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1057/2152] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:02:55 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:02:55.794581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:02:55.796249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:02:55.848150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:02:55.854503 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332f0937-0524-40b0-b4e5-cd9bde15882b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:02:55.858918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e822e0-1c2c-4203-9e67-da5dddfcd9f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:02:55.859513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e822e0-1c2c-4203-9e67-da5dddfcd9f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:02:55.859789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e822e0-1c2c-4203-9e67-da5dddfcd9f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:02:55.860054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e822e0-1c2c-4203-9e67-da5dddfcd9f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:02:55.868101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e822e0-1c2c-4203-9e67-da5dddfcd9f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:02:55.869585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e822e0-1c2c-4203-9e67-da5dddfcd9f9 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:02:55.878878 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1096/2153] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:02:55 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:03:00.934875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055499e-1847-4e3c-ae1c-c80e007a0b77 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:00.995196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3055499e-1847-4e3c-ae1c-c80e007a0b77 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:00.999676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:01.000410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:01.000691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:01.000962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] PATH_INFO: `/v3/users/9748a495a8c044c0a4470a7d3afeecb0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:01.002102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] RBAC: Authorizing `identity:delete_user(user_id=9748a495a8c044c0a4470a7d3afeecb0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:01.004272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:01.036633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:03:01.058389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9748a495a8c044c0a4470a7d3afeecb0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.059592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9748a495a8c044c0a4470a7d3afeecb0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.067813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9748a495a8c044c0a4470a7d3afeecb0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.081333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9748a495a8c044c0a4470a7d3afeecb0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.081980 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9748a495a8c044c0a4470a7d3afeecb0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.096366 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1058/2154] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:03:00 2022] DELETE /identity/v3/users/9748a495a8c044c0a4470a7d3afeecb0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:03:01.101202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:01.101788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:01.101788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:01.124596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:03:01.125828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:01.190001 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1097/2155] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:03:01.200124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:01.201721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:01.265504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4a130-0f54-4cb2-b069-5f89cf4fbeac admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:01.269278 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd6f2b-c14f-4c42-8ec1-43723081b45f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:01.269991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd6f2b-c14f-4c42-8ec1-43723081b45f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:01.270212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd6f2b-c14f-4c42-8ec1-43723081b45f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:01.270433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd6f2b-c14f-4c42-8ec1-43723081b45f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:01.280561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd6f2b-c14f-4c42-8ec1-43723081b45f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:01.281985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd6f2b-c14f-4c42-8ec1-43723081b45f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:01.294062 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1059/2156] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:03:01.681863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:01.683365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:01.696231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef63ee-f719-4452-bdec-528f6f7d516f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:01.703692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:01.704807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:01.704807 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:01.705024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] PATH_INFO: `/v3/projects/79cd0f930dca4501b35b200777cee537` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:01.706005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79cd0f930dca4501b35b200777cee537)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:01.708033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:01.755173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79cd0f930dca4501b35b200777cee537', 'request_id': 'req-5f890174-b151-444d-91f9-92c41b1c37a9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.762946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79cd0f930dca4501b35b200777cee537', 'request_id': 'req-5f890174-b151-444d-91f9-92c41b1c37a9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.769140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79cd0f930dca4501b35b200777cee537', 'request_id': 'req-5f890174-b151-444d-91f9-92c41b1c37a9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.780174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] The token cache is being invalidate because project 79cd0f930dca4501b35b200777cee537 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:03:01.780487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.781130 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:03:01.783342 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1098/2157] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:03:01 2022] DELETE /identity/v3/projects/79cd0f930dca4501b35b200777cee537 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:03:09.717328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:09.717946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:09.718190 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:09.739130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:03:09.740339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:09.798048 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1060/2158] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 01:03:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:03:09.854745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:09.856166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:09.925487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f890174-b151-444d-91f9-92c41b1c37a9 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:09.930510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5b794f-fa8b-46f0-8ca0-1bed054db14f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:09.931114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b794f-fa8b-46f0-8ca0-1bed054db14f service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:09.931393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b794f-fa8b-46f0-8ca0-1bed054db14f service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:09.931687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5b794f-fa8b-46f0-8ca0-1bed054db14f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:09.940424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b794f-fa8b-46f0-8ca0-1bed054db14f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:09.942002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5b794f-fa8b-46f0-8ca0-1bed054db14f service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:09.950210 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1099/2159] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:03:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:03:19.790968 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=70,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:03:19.946905 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:03:23.936194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:23.937648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:24.076129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cd26fe-96f3-4580-8071-e30fb0522fd4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:24.081240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff53e41-a172-47b3-a1ca-c6fa4843d02d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:24.081784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff53e41-a172-47b3-a1ca-c6fa4843d02d service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:24.082060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff53e41-a172-47b3-a1ca-c6fa4843d02d service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:24.082331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff53e41-a172-47b3-a1ca-c6fa4843d02d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:24.084045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bff53e41-a172-47b3-a1ca-c6fa4843d02d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:24.095651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff53e41-a172-47b3-a1ca-c6fa4843d02d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:24.097238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff53e41-a172-47b3-a1ca-c6fa4843d02d service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:24.106849 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1061/2160] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:03:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:03:26.013494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144248-cb76-4a58-918a-2d1857e9c8cd None None] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:26.014002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144248-cb76-4a58-918a-2d1857e9c8cd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:26.014129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144248-cb76-4a58-918a-2d1857e9c8cd None None] PATH_INFO: `/` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:26.015453 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1100/2161] 10.222.0.39 () {54 vars in 921 bytes} [Sun Oct 23 01:03:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:03:26.020239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252c230-7079-47c8-8680-bd6125b87fb3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:26.020688 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252c230-7079-47c8-8680-bd6125b87fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:26.020989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c252c230-7079-47c8-8680-bd6125b87fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:26.074045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c252c230-7079-47c8-8680-bd6125b87fb3 None None] MFA Rules not processed for user `f04fb9e1d1bd42ac815078b61f6a17f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:03:26.075277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c252c230-7079-47c8-8680-bd6125b87fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:26.122336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:03:26.159779 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1062/2162] 10.222.0.39 () {58 vars in 1018 bytes} [Sun Oct 23 01:03:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:03:26.171833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c144248-cb76-4a58-918a-2d1857e9c8cd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:26.173563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6c144248-cb76-4a58-918a-2d1857e9c8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:26.229541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c144248-cb76-4a58-918a-2d1857e9c8cd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:26.233123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:26.233709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:26.233982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:26.234246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:26.244051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:26.245486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:26.255924 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1101/2163] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:03:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:03:28.153856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68dff1-2c5e-4526-84d1-ae91b9b6f153 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:28.154327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68dff1-2c5e-4526-84d1-ae91b9b6f153 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:28.154510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68dff1-2c5e-4526-84d1-ae91b9b6f153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:28.173848 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1b68dff1-2c5e-4526-84d1-ae91b9b6f153 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:03:28.175021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1b68dff1-2c5e-4526-84d1-ae91b9b6f153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:28.181976 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1063/2164] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 01:03:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:03:28.194173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:28.205432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8cd1f-5b7f-4401-971c-651090f40e85 service glance] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:28.210412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:28.211029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:28.211304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:28.211571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:28.220217 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:28.221676 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:28.230778 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1102/2165] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:03:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:03:38.168583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=17 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:03:38.226679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=32 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:03:50.013309 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:50.013812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:50.014172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:50.034680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:03:50.035854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:03:50.042863 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1064/2166] 10.222.0.39 () {58 vars in 967 bytes} [Sun Oct 23 01:03:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:03:50.055837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:03:50.066164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0889216d-99d1-47c9-88f1-816b1dd07d55 service swift] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:03:50.071131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:03:50.071722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:03:50.072002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:03:50.072276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:03:50.080101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:03:50.081774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:03:50.089692 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1103/2167] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:03:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:04:00.029398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:00.086502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:04.077338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:04.078732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:04.155639 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa15c6-1009-4e12-83e0-a3127c211c17 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:04.159071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:04.159667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:04.159943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:04.160218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:04.161979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:04.243023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:04.244463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:04.262853 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1065/2168] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:07.421992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:07.445799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21509364-c191-48dc-b79c-1ff9138016a0 service swift] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:07.450853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:07.451497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:07.451790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:07.452057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:07.453985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:07.466291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:07.467890 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:07.478486 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1104/2169] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:07.713243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:07.715146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:07.727914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4eba9-b043-4e38-b27d-831a22e1d78f service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:07.733244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:07.733875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:07.734155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:07.734428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] PATH_INFO: `/v3/users/992874ab1b9c4fce90b46e659910d512` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:07.735553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=992874ab1b9c4fce90b46e659910d512)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:07.737533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:07.783922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992874ab1b9c4fce90b46e659910d512'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:07.784662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992874ab1b9c4fce90b46e659910d512'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:07.791403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992874ab1b9c4fce90b46e659910d512'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:07.792499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:07.806578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992874ab1b9c4fce90b46e659910d512'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:07.808163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-51677913-984d-43cd-9076-9870ba39a53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992874ab1b9c4fce90b46e659910d512'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:07.823971 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1066/2170] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:07 2022] DELETE /identity/v3/users/992874ab1b9c4fce90b46e659910d512 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:07.989968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.002563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d19c2a-862a-400c-88d4-16fc7a59d981 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.069123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.069833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.070120 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.070392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] PATH_INFO: `/v3/projects/11730d9eec8546c5aa486c1a4a283bf8` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.071713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11730d9eec8546c5aa486c1a4a283bf8)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.073671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.122021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11730d9eec8546c5aa486c1a4a283bf8', 'request_id': 'req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:08.129662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11730d9eec8546c5aa486c1a4a283bf8', 'request_id': 'req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:08.136074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11730d9eec8546c5aa486c1a4a283bf8', 'request_id': 'req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:08.146622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] The token cache is being invalidate because project 11730d9eec8546c5aa486c1a4a283bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:08.146900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:08.147505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-45eb1d7d-5dbd-4ec7-b81e-1dbbcc060151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:08.148995 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1105/2171] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:07 2022] DELETE /identity/v3/projects/11730d9eec8546c5aa486c1a4a283bf8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:08.154363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf2ec-095b-4cb8-8525-db6d91893c09 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.154645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf2ec-095b-4cb8-8525-db6d91893c09 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.154915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf2ec-095b-4cb8-8525-db6d91893c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.201246 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0ffcf2ec-095b-4cb8-8525-db6d91893c09 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:08.202355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0ffcf2ec-095b-4cb8-8525-db6d91893c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:08.258690 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1067/2172] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:08.266963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225cf1-d6e3-41b2-9d6b-45245ad15fe3 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.267242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225cf1-d6e3-41b2-9d6b-45245ad15fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.267509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28225cf1-d6e3-41b2-9d6b-45245ad15fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.287458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-28225cf1-d6e3-41b2-9d6b-45245ad15fe3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:08.288766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-28225cf1-d6e3-41b2-9d6b-45245ad15fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:08.297186 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1106/2173] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:08.301244 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcf2ec-095b-4cb8-8525-db6d91893c09 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.311198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcf2ec-095b-4cb8-8525-db6d91893c09 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.315599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.316186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.316488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.316754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.317381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:08.317676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.319232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.330597 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1068/2174] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:04:08.334696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225cf1-d6e3-41b2-9d6b-45245ad15fe3 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.343725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28225cf1-d6e3-41b2-9d6b-45245ad15fe3 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.348361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.348950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.349229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.349502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.350137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.352012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.379318 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1107/2175] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:08 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:04:08.383367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.393085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6801fdfc-cbe3-44a6-bd37-f4854d761948 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.455491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.456019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.456320 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.456597 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.457258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.459165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.484464 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1069/2176] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:08 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:08.485640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.495937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8007a-4366-4df4-90d0-a5cb839d56c6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.501242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.501835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.502115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.502388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.503027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.504954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.514952 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1108/2177] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:08.518732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.528821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5b3a9d-6e34-4a47-9a75-4d7a9ea68189 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.533594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb4cbbf-ffde-408a-9456-f9d530dbd2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.534213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4cbbf-ffde-408a-9456-f9d530dbd2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.534489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4cbbf-ffde-408a-9456-f9d530dbd2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.534758 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb4cbbf-ffde-408a-9456-f9d530dbd2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.535388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4cbbf-ffde-408a-9456-f9d530dbd2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.536994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb4cbbf-ffde-408a-9456-f9d530dbd2e6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.548172 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1070/2178] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:08.551837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.561425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe598f1-9467-494c-b121-6f2264b98319 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.566043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.566687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.566963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.567235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] PATH_INFO: `/v3/projects/847fda4d166049ee8bcb1039575e3717/users/280fe650f1454227a6c9086349d292cc/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.581675 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] RBAC: Authorizing `identity:create_grant(project_id=847fda4d166049ee8bcb1039575e3717, user_id=280fe650f1454227a6c9086349d292cc, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.584343 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.598264 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1109/2179] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:08 2022] PUT /identity/v3/projects/847fda4d166049ee8bcb1039575e3717/users/280fe650f1454227a6c9086349d292cc/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:08.609847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.610127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.610422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.637855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] MFA Rules not processed for user `280fe650f1454227a6c9086349d292cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:08.639059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:08.643907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:08.706062 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1071/2180] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:08.717381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:08.719034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:08.737215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:08.799024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662c347-cd7a-4d9c-bda8-20d83a5ed00b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:08.802826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:08.803429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:08.803706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:08.803978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:08.816079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:08.817524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:08.826367 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1110/2181] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:13.498896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:13.500387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:13.558683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbebdf3-57cb-47ad-a271-96a6fc4dee88 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:13.562200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae559b1d-2b00-4c8d-ac53-cbba22b45ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:13.562804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae559b1d-2b00-4c8d-ac53-cbba22b45ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:13.563081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae559b1d-2b00-4c8d-ac53-cbba22b45ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:13.563350 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae559b1d-2b00-4c8d-ac53-cbba22b45ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:13.565106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-ae559b1d-2b00-4c8d-ac53-cbba22b45ff8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:13.624809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae559b1d-2b00-4c8d-ac53-cbba22b45ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:13.626151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae559b1d-2b00-4c8d-ac53-cbba22b45ff8 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:13.635976 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1072/2182] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:17.308671 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:17.369669 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2b9e2c-804e-4f31-9702-f60b503f0132 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:17.374594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:17.375210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:17.375488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:17.375765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] PATH_INFO: `/v3/users/280fe650f1454227a6c9086349d292cc` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:17.376877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] RBAC: Authorizing `identity:delete_user(user_id=280fe650f1454227a6c9086349d292cc)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:17.379042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:17.425910 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '280fe650f1454227a6c9086349d292cc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:17.433429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '280fe650f1454227a6c9086349d292cc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:17.441312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '280fe650f1454227a6c9086349d292cc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:17.453629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '280fe650f1454227a6c9086349d292cc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:17.454520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '280fe650f1454227a6c9086349d292cc'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:17.460663 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1111/2183] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:17 2022] DELETE /identity/v3/users/280fe650f1454227a6c9086349d292cc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:17.465149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:17.465644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:17.465757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:17.484932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:17.486135 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:17.548873 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1073/2184] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:17.558504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:17.559966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:17.578321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:17.619007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3214be-61da-4bab-aa99-a06777096608 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:17.622539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d276e3-82f7-496e-850f-fc844c52d6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:17.623132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d276e3-82f7-496e-850f-fc844c52d6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:17.623406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d276e3-82f7-496e-850f-fc844c52d6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:17.623674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d276e3-82f7-496e-850f-fc844c52d6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:17.632511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d276e3-82f7-496e-850f-fc844c52d6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:17.633936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d276e3-82f7-496e-850f-fc844c52d6b8 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:17.642954 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1112/2185] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:17.910186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:17.911675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:17.919438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:17.925281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe838757-4fb5-4459-bbb2-ebe9f31ee1e3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:17.929793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:17.930432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:17.930708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:17.930973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] PATH_INFO: `/v3/projects/847fda4d166049ee8bcb1039575e3717` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:17.932082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=847fda4d166049ee8bcb1039575e3717)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:17.934794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:17.982756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '847fda4d166049ee8bcb1039575e3717', 'request_id': 'req-064f7179-a37f-4eff-baf3-9fe69b65f029'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:17.989708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '847fda4d166049ee8bcb1039575e3717', 'request_id': 'req-064f7179-a37f-4eff-baf3-9fe69b65f029'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:17.995707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '847fda4d166049ee8bcb1039575e3717', 'request_id': 'req-064f7179-a37f-4eff-baf3-9fe69b65f029'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:18.006707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] The token cache is being invalidate because project 847fda4d166049ee8bcb1039575e3717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:18.006989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:18.007611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-064f7179-a37f-4eff-baf3-9fe69b65f029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:18.009258 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1074/2186] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:17 2022] DELETE /identity/v3/projects/847fda4d166049ee8bcb1039575e3717 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:18.014521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe23bb-caa0-482e-984c-edcacc333aef None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.014806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe23bb-caa0-482e-984c-edcacc333aef None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.015081 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe23bb-caa0-482e-984c-edcacc333aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.034469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0fbe23bb-caa0-482e-984c-edcacc333aef None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:18.035650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0fbe23bb-caa0-482e-984c-edcacc333aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:18.092930 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1113/2187] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:18.100585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866188f3-3b7e-4b68-8d8e-5e54c604e871 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.100866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866188f3-3b7e-4b68-8d8e-5e54c604e871 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.101134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866188f3-3b7e-4b68-8d8e-5e54c604e871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.125001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-866188f3-3b7e-4b68-8d8e-5e54c604e871 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:18.126188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-866188f3-3b7e-4b68-8d8e-5e54c604e871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:18.135974 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1075/2188] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:18.141476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe23bb-caa0-482e-984c-edcacc333aef None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.150840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe23bb-caa0-482e-984c-edcacc333aef None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.155292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.155886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.156188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.156528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.157174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:18.157471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.159058 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.174092 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1114/2189] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:04:18.175279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866188f3-3b7e-4b68-8d8e-5e54c604e871 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.184942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866188f3-3b7e-4b68-8d8e-5e54c604e871 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.189500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.190119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.190389 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.190653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.191276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.193183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.220341 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1076/2190] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:18.224201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.234530 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758f5c0b-d194-408c-a114-1a757cd74a18 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.293182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.294294 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.294569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.294838 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.295481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.305709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.335042 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1115/2191] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/users => generated 282 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:18.336175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.349419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.349588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a365e064-64bf-4e33-8c4d-1724f20d6c6d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.351079 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:18.354245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.354894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.355172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.355441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.356747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.358346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.369552 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1077/2192] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:18.373585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.383553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d161e2-8097-45b1-a481-3e9a4d60945a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.388173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.388784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.389059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.389331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.389987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.390991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:18.391513 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.404979 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1078/2193] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:18.407012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.410827 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ad37df-d460-48f3-ac06-b4d961d259ac admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.414491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76e15a-e234-4248-9ea2-5059fece2b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.415085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e15a-e234-4248-9ea2-5059fece2b04 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.415387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e15a-e234-4248-9ea2-5059fece2b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.415662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76e15a-e234-4248-9ea2-5059fece2b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.416380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d8a55-bdb2-4524-8dcc-77655189e5c4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.417418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e76e15a-e234-4248-9ea2-5059fece2b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:18.421435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.422066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.422345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.422615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] PATH_INFO: `/v3/projects/3c58193cf8f34618963845f2bb38522f/users/9f72dfc419cf4f9e94af1a98d74c62bc/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.429693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76e15a-e234-4248-9ea2-5059fece2b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.431106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76e15a-e234-4248-9ea2-5059fece2b04 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.437297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c58193cf8f34618963845f2bb38522f, user_id=9f72dfc419cf4f9e94af1a98d74c62bc, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.439917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.440806 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1116/2194] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:04:18.454861 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1079/2195] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:18 2022] PUT /identity/v3/projects/3c58193cf8f34618963845f2bb38522f/users/9f72dfc419cf4f9e94af1a98d74c62bc/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:18.467173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cb97c-4ffb-48f6-870e-5a310689a756 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.467457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cb97c-4ffb-48f6-870e-5a310689a756 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.467753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cb97c-4ffb-48f6-870e-5a310689a756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.496584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e99cb97c-4ffb-48f6-870e-5a310689a756 None None] MFA Rules not processed for user `9f72dfc419cf4f9e94af1a98d74c62bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:18.497917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e99cb97c-4ffb-48f6-870e-5a310689a756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:18.563319 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1117/2196] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:18.567583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.636211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdff006-793d-459c-ac7a-7506ecae2787 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.641260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.641869 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.642147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.642410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.643034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.644958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.662071 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:18.678566 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1080/2197] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:18.682735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cb97c-4ffb-48f6-870e-5a310689a756 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.693424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cb97c-4ffb-48f6-870e-5a310689a756 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.751758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.752413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.752691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.752960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.753845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.755742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.782034 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1118/2198] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:18.783517 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.793847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16863e9a-c226-44b6-ade9-1e83a61379bf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.798508 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.799106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.799380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.799645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.800681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.801870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.816336 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1081/2199] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:18.820203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.830848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dde440-303c-45e0-9859-7f9f4c500690 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.835724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.836419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.836694 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.836961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] PATH_INFO: `/v3/projects/bdb16dbd945c4b0bb6e0079948d8be01/users/fd15d45a2e3f4724a52ecaf62b9d241b/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.853467 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb16dbd945c4b0bb6e0079948d8be01, user_id=fd15d45a2e3f4724a52ecaf62b9d241b, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.856240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.869751 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1119/2200] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:18 2022] PUT /identity/v3/projects/bdb16dbd945c4b0bb6e0079948d8be01/users/fd15d45a2e3f4724a52ecaf62b9d241b/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:18.873677 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.883835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75893386-58e3-4daf-8452-cd5e082720a0 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.940403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c61e901-df9f-4358-9f23-e7b099a7dbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.940928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61e901-df9f-4358-9f23-e7b099a7dbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.941210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61e901-df9f-4358-9f23-e7b099a7dbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.941482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c61e901-df9f-4358-9f23-e7b099a7dbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.942150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61e901-df9f-4358-9f23-e7b099a7dbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.943742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c61e901-df9f-4358-9f23-e7b099a7dbe6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.954132 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1082/2201] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:18.958057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:18.968723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79c4db6-8b19-43ea-b94f-1c0d11923b8c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:18.973324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:18.973971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:18.974252 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:18.974528 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] PATH_INFO: `/v3/projects/bdb16dbd945c4b0bb6e0079948d8be01/users/fd15d45a2e3f4724a52ecaf62b9d241b/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:18.976457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb16dbd945c4b0bb6e0079948d8be01, user_id=fd15d45a2e3f4724a52ecaf62b9d241b, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:18.979087 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:18.989710 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1120/2202] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:18 2022] PUT /identity/v3/projects/bdb16dbd945c4b0bb6e0079948d8be01/users/fd15d45a2e3f4724a52ecaf62b9d241b/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:19.000455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc7936-c792-4f80-bfd2-7fb7aaff20d6 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:19.000738 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc7936-c792-4f80-bfd2-7fb7aaff20d6 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:19.001010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc7936-c792-4f80-bfd2-7fb7aaff20d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:19.029152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-19bc7936-c792-4f80-bfd2-7fb7aaff20d6 None None] MFA Rules not processed for user `fd15d45a2e3f4724a52ecaf62b9d241b`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:19.030387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-19bc7936-c792-4f80-bfd2-7fb7aaff20d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:19.099113 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1083/2203] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:19.111001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:19.112881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:19.135448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:19.174518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc131d66-ece7-4ea0-90bb-e7b362e0b985 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:19.178327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdb728-7b89-42d7-89e4-9c88450a0670 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:19.178926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdb728-7b89-42d7-89e4-9c88450a0670 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:19.179202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdb728-7b89-42d7-89e4-9c88450a0670 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:19.179479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdb728-7b89-42d7-89e4-9c88450a0670 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:19.188527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cdb728-7b89-42d7-89e4-9c88450a0670 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:19.189990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cdb728-7b89-42d7-89e4-9c88450a0670 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:19.200743 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1121/2204] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:19 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:19.674405 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc7936-c792-4f80-bfd2-7fb7aaff20d6 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:19.763672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bc7936-c792-4f80-bfd2-7fb7aaff20d6 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:19.769828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:19.770087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:19.770364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:19.770632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] PATH_INFO: `/v3/users/9f72dfc419cf4f9e94af1a98d74c62bc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:19.771788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f72dfc419cf4f9e94af1a98d74c62bc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:19.773745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:19.783347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:19.848235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f72dfc419cf4f9e94af1a98d74c62bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:19.849648 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f72dfc419cf4f9e94af1a98d74c62bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:19.858048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f72dfc419cf4f9e94af1a98d74c62bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:19.873516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f72dfc419cf4f9e94af1a98d74c62bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:19.873516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f72dfc419cf4f9e94af1a98d74c62bc'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:19.894900 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1084/2205] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:19 2022] DELETE /identity/v3/users/9f72dfc419cf4f9e94af1a98d74c62bc => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:19.896652 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:19.897105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:19.897340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:19.921474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:19.922707 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:19.994777 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1122/2206] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:20.005448 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:20.007500 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:20.135511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508e2a-9973-4a2b-ba4f-e11a8958f317 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:20.140373 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:20.141013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:20.141408 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:20.141684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:20.153784 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:20.155260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:20.167046 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1085/2207] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:20.592567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:20.595025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:20.626432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3d64b-6a01-4a12-984d-b33aabde8f42 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:20.641140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:20.641954 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:20.642351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:20.642724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] PATH_INFO: `/v3/projects/3c58193cf8f34618963845f2bb38522f` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:20.645561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c58193cf8f34618963845f2bb38522f)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:20.647614 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:20.740044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c58193cf8f34618963845f2bb38522f', 'request_id': 'req-6532ea00-f7c0-4d61-b583-46ce4da23be5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.750409 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c58193cf8f34618963845f2bb38522f', 'request_id': 'req-6532ea00-f7c0-4d61-b583-46ce4da23be5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.758158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c58193cf8f34618963845f2bb38522f', 'request_id': 'req-6532ea00-f7c0-4d61-b583-46ce4da23be5'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.768983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] The token cache is being invalidate because project 3c58193cf8f34618963845f2bb38522f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:20.769266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.769907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.771369 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1123/2208] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:20 2022] DELETE /identity/v3/projects/3c58193cf8f34618963845f2bb38522f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:20.775531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:20.785242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7181f11-1a08-45ac-91f8-67aad5354bd2 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:20.851716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:20.852380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:20.852654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:20.852950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] PATH_INFO: `/v3/users/fd15d45a2e3f4724a52ecaf62b9d241b` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:20.854060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd15d45a2e3f4724a52ecaf62b9d241b)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:20.856031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:20.892966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:20.909507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd15d45a2e3f4724a52ecaf62b9d241b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.910234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd15d45a2e3f4724a52ecaf62b9d241b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.917270 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd15d45a2e3f4724a52ecaf62b9d241b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.930052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd15d45a2e3f4724a52ecaf62b9d241b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.930716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-3b57c25d-130b-4676-b6bf-e2cb9e577293 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd15d45a2e3f4724a52ecaf62b9d241b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:20.944108 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1086/2209] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:20 2022] DELETE /identity/v3/users/fd15d45a2e3f4724a52ecaf62b9d241b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:21.369524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:21.372077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:21.389820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6532ea00-f7c0-4d61-b583-46ce4da23be5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:21.449106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:21.466937 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:21.467656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.468003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.468372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] PATH_INFO: `/v3/projects/bdb16dbd945c4b0bb6e0079948d8be01` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.469596 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb16dbd945c4b0bb6e0079948d8be01)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:21.471570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:21.543684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb16dbd945c4b0bb6e0079948d8be01', 'request_id': 'req-9ccd39d1-4899-41a9-b3ec-722fea113622'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:21.553571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb16dbd945c4b0bb6e0079948d8be01', 'request_id': 'req-9ccd39d1-4899-41a9-b3ec-722fea113622'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:21.561203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb16dbd945c4b0bb6e0079948d8be01', 'request_id': 'req-9ccd39d1-4899-41a9-b3ec-722fea113622'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:21.571995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] The token cache is being invalidate because project bdb16dbd945c4b0bb6e0079948d8be01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:21.572374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:21.573164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9ccd39d1-4899-41a9-b3ec-722fea113622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:21.575467 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1124/2210] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:21 2022] DELETE /identity/v3/projects/bdb16dbd945c4b0bb6e0079948d8be01 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:21.580905 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e3272-b44f-4b0e-981b-d914d2f87c32 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.581275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e3272-b44f-4b0e-981b-d914d2f87c32 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.581610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0e3272-b44f-4b0e-981b-d914d2f87c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.601982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5d0e3272-b44f-4b0e-981b-d914d2f87c32 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:21.603236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5d0e3272-b44f-4b0e-981b-d914d2f87c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:21.664091 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1087/2211] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:21.672473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7665a-f777-4f12-a4fc-47323eaae8ff None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.672828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7665a-f777-4f12-a4fc-47323eaae8ff None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.673167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7665a-f777-4f12-a4fc-47323eaae8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.694286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9dc7665a-f777-4f12-a4fc-47323eaae8ff None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:21.695486 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9dc7665a-f777-4f12-a4fc-47323eaae8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:21.708618 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1125/2212] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:21.713125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e3272-b44f-4b0e-981b-d914d2f87c32 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:21.723949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0e3272-b44f-4b0e-981b-d914d2f87c32 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:21.732066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:21.735714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.735714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.735714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.740600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:21.740600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:21.741777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:21.759434 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1088/2213] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:04:21.760860 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7665a-f777-4f12-a4fc-47323eaae8ff None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:21.773634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7665a-f777-4f12-a4fc-47323eaae8ff None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:21.781549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:21.782361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.782656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.783014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.783772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:21.786072 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:21.829149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:21.832651 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1126/2214] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:21 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:21.839435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32a5c3-f6e7-4ccb-9bfd-900cf80a6279 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:21.896643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:21.897251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.897527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.897793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.898414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:21.900315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:21.922562 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1089/2215] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:21 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:04:21.926650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:21.938151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e0f22-64d3-4913-abec-75b9acf11117 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:21.944354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:21.945038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.945411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.945751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.946474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:21.948344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:21.959046 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1127/2216] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:21.963052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:21.972983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4b472-ebbd-46d0-b7f6-953814647e22 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:21.977815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705b57a9-b034-4248-9f27-42ee456b1cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:21.978458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b57a9-b034-4248-9f27-42ee456b1cdb admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:21.978702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b57a9-b034-4248-9f27-42ee456b1cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:21.978973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705b57a9-b034-4248-9f27-42ee456b1cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:21.979619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b57a9-b034-4248-9f27-42ee456b1cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:21.981797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705b57a9-b034-4248-9f27-42ee456b1cdb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:21.999126 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1090/2217] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:21.999924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.010601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff7226-9a9a-4fdd-987f-adf1e0023fbf admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.015462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.016117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.016445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.016769 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] PATH_INFO: `/v3/projects/2bc34cfbc44b4565beda49562c510fe4/users/b4e1d29bb7a5421b8570b02b3eeaebf0/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.047347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bc34cfbc44b4565beda49562c510fe4, user_id=b4e1d29bb7a5421b8570b02b3eeaebf0, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.050177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.069661 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1128/2218] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:21 2022] PUT /identity/v3/projects/2bc34cfbc44b4565beda49562c510fe4/users/b4e1d29bb7a5421b8570b02b3eeaebf0/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:22.084496 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f4a5f-a557-4355-b951-278e861831e4 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.084783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f4a5f-a557-4355-b951-278e861831e4 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.085059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f4a5f-a557-4355-b951-278e861831e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.123588 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:22.127197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3b9f4a5f-a557-4355-b951-278e861831e4 None None] MFA Rules not processed for user `b4e1d29bb7a5421b8570b02b3eeaebf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:22.129452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3b9f4a5f-a557-4355-b951-278e861831e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:22.200587 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1091/2219] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:22.205021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.274313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038ec4-8c42-4bb0-a933-1d7b1de6212d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.280422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.281033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.281313 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.281587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.282222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.284115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.306774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:22.317280 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1129/2220] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:22 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:22.319124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f4a5f-a557-4355-b951-278e861831e4 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.331225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f4a5f-a557-4355-b951-278e861831e4 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.390582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.391194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.391475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.391749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.392421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.394324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.420585 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1092/2221] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:22 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:22.422750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.432914 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158af9e-ad0e-41c8-ab6c-20970890cbf6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.437491 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.438085 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.438363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.438628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.439265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.440872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.451834 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1130/2222] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:22.456209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.466580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068981f8-e521-415a-920c-5b18833ffd0a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.471030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.471684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.471960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.472230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] PATH_INFO: `/v3/projects/f6ec7fc24b934d599380c122eaf8ae58/users/853e254d4efd48ada7cd9963c7be9321/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.487120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ec7fc24b934d599380c122eaf8ae58, user_id=853e254d4efd48ada7cd9963c7be9321, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.489778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.503946 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1093/2223] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:22 2022] PUT /identity/v3/projects/f6ec7fc24b934d599380c122eaf8ae58/users/853e254d4efd48ada7cd9963c7be9321/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:22.508234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.518749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29615a3b-914b-431e-bc53-dd49200ff76d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.576800 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36bff45-5c6a-4448-a50f-73ab15ffa840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.577315 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bff45-5c6a-4448-a50f-73ab15ffa840 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.577595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bff45-5c6a-4448-a50f-73ab15ffa840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.577863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36bff45-5c6a-4448-a50f-73ab15ffa840 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.578499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bff45-5c6a-4448-a50f-73ab15ffa840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.580069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36bff45-5c6a-4448-a50f-73ab15ffa840 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.590417 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1131/2224] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:22 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:22.594701 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.604986 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7618b-f097-4a49-9bbf-7ab60bdce0b2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.609637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.610292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.610569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.610841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] PATH_INFO: `/v3/projects/f6ec7fc24b934d599380c122eaf8ae58/users/853e254d4efd48ada7cd9963c7be9321/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.612827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6ec7fc24b934d599380c122eaf8ae58, user_id=853e254d4efd48ada7cd9963c7be9321, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.615450 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.625758 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1094/2225] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:22 2022] PUT /identity/v3/projects/f6ec7fc24b934d599380c122eaf8ae58/users/853e254d4efd48ada7cd9963c7be9321/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:22.636116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.636431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.637236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.672690 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] MFA Rules not processed for user `853e254d4efd48ada7cd9963c7be9321`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:22.673892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:22.740772 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1132/2226] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:22.752428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.754590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:22.812090 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babcef7a-5c35-4df5-849b-e1174d937348 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.815995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.816704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.816985 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.817259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.828266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.829653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.835706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:22.844386 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1095/2227] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:22 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:22.889151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:22.890568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:22.903681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53fe86b-5117-4d23-bee5-ab325fe25e6f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:22.909881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:22.910752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:22.911089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:22.911425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:22.982872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:22.984485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:22.994399 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1133/2228] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:25.338910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:25.340495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:25.405293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b956b-52a7-4b66-992b-973f5ecf575d service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:25.409778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:25.410399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:25.410674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:25.410942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] PATH_INFO: `/v3/users/51c96b11cfe540debe7e32c1d5b915a7` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:25.412059 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c96b11cfe540debe7e32c1d5b915a7)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:25.414013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:25.461499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c96b11cfe540debe7e32c1d5b915a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.462381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c96b11cfe540debe7e32c1d5b915a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.469348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c96b11cfe540debe7e32c1d5b915a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.480739 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c96b11cfe540debe7e32c1d5b915a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.481871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c96b11cfe540debe7e32c1d5b915a7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.494018 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1096/2229] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:25 2022] DELETE /identity/v3/users/51c96b11cfe540debe7e32c1d5b915a7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:25.661882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:25.673361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649cf7f-6016-42c6-a0d6-1b1831b96f2f service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:25.677002 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:25.730468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:25.731098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:25.731957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:25.732240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] PATH_INFO: `/v3/projects/ac18f8515a94410891c7675051d1ac59` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:25.733381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac18f8515a94410891c7675051d1ac59)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:25.735280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:25.779556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac18f8515a94410891c7675051d1ac59', 'request_id': 'req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.786555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac18f8515a94410891c7675051d1ac59', 'request_id': 'req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.793169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac18f8515a94410891c7675051d1ac59', 'request_id': 'req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.803997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] The token cache is being invalidate because project ac18f8515a94410891c7675051d1ac59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:25.804284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.804936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.806391 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1134/2230] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:25 2022] DELETE /identity/v3/projects/ac18f8515a94410891c7675051d1ac59 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:25.810567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:25.812021 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:25.825109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c7e603-b468-4415-ab47-6e7d99bfd3fd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:25.881833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:25.882473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:25.882749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:25.883017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] PATH_INFO: `/v3/users/fef5184f076e43a8bbf058b99467c2f5` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:25.884132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fef5184f076e43a8bbf058b99467c2f5)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:25.886062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:25.907838 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:25.947492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fef5184f076e43a8bbf058b99467c2f5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.947492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fef5184f076e43a8bbf058b99467c2f5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.951691 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fef5184f076e43a8bbf058b99467c2f5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.964171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fef5184f076e43a8bbf058b99467c2f5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.964877 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fef5184f076e43a8bbf058b99467c2f5'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:25.979163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1097/2231] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:25 2022] DELETE /identity/v3/users/fef5184f076e43a8bbf058b99467c2f5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:26.178504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:26.180064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:26.194496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26e8ea-459c-42c9-8c27-d7a5f15ca2ee admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:26.261678 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:26.262303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:26.263110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:26.263431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] PATH_INFO: `/v3/projects/99d6804d5c474f00ab3abbceab6f4165` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:26.264588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99d6804d5c474f00ab3abbceab6f4165)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:26.266479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:26.309508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:26.318790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99d6804d5c474f00ab3abbceab6f4165', 'request_id': 'req-7032a546-082d-4365-b98a-229bd5a4a8c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.327181 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99d6804d5c474f00ab3abbceab6f4165', 'request_id': 'req-7032a546-082d-4365-b98a-229bd5a4a8c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.335096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99d6804d5c474f00ab3abbceab6f4165', 'request_id': 'req-7032a546-082d-4365-b98a-229bd5a4a8c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.348832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] The token cache is being invalidate because project 99d6804d5c474f00ab3abbceab6f4165 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:26.349271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.349975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7032a546-082d-4365-b98a-229bd5a4a8c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.351767 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1135/2232] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:26 2022] DELETE /identity/v3/projects/99d6804d5c474f00ab3abbceab6f4165 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:26.488285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:26.489845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:26.504880 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da18ea9-305d-4d90-b106-923a17365602 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:26.578487 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:26.579150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:26.579432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:26.580332 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] PATH_INFO: `/v3/users/b4e1d29bb7a5421b8570b02b3eeaebf0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:26.581529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4e1d29bb7a5421b8570b02b3eeaebf0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:26.583658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:26.640667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4e1d29bb7a5421b8570b02b3eeaebf0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.641323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4e1d29bb7a5421b8570b02b3eeaebf0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.648746 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4e1d29bb7a5421b8570b02b3eeaebf0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.661779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4e1d29bb7a5421b8570b02b3eeaebf0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.662397 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4e1d29bb7a5421b8570b02b3eeaebf0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:26.677052 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1098/2233] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:26 2022] DELETE /identity/v3/users/b4e1d29bb7a5421b8570b02b3eeaebf0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:26.681659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:26.682138 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:26.682479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:26.702882 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:26.704155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:26.764863 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1136/2234] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:26.774504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:26.775896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:26.795327 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:26.840352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a7bdc-6b6d-405e-bd78-ee67cd844167 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:26.844586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:26.845272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:26.845620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:26.845946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:26.855137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:26.857213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:26.866948 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1099/2235] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:27.139289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:27.140850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:27.154030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e10ad6-d8de-4ebf-9d88-9865a1987f8b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:27.158498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:27.159124 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:27.159399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:27.160166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] PATH_INFO: `/v3/projects/2bc34cfbc44b4565beda49562c510fe4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:27.161422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bc34cfbc44b4565beda49562c510fe4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:27.163384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:27.209213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bc34cfbc44b4565beda49562c510fe4', 'request_id': 'req-0ac5643f-0847-4333-b9e4-8ce836630862'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.216712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bc34cfbc44b4565beda49562c510fe4', 'request_id': 'req-0ac5643f-0847-4333-b9e4-8ce836630862'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.222826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bc34cfbc44b4565beda49562c510fe4', 'request_id': 'req-0ac5643f-0847-4333-b9e4-8ce836630862'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.233527 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] The token cache is being invalidate because project 2bc34cfbc44b4565beda49562c510fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:27.233808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.234418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.235880 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1137/2236] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:27 2022] DELETE /identity/v3/projects/2bc34cfbc44b4565beda49562c510fe4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:27.239856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:27.249583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4de47e-1400-4a57-bf0a-2f365e5f8f0f service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:27.304683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:27.305307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:27.305586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:27.305857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] PATH_INFO: `/v3/users/853e254d4efd48ada7cd9963c7be9321` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:27.306967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=853e254d4efd48ada7cd9963c7be9321)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:27.309110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:27.353936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853e254d4efd48ada7cd9963c7be9321'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.354577 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853e254d4efd48ada7cd9963c7be9321'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.362247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853e254d4efd48ada7cd9963c7be9321'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.373871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853e254d4efd48ada7cd9963c7be9321'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.374509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-942babb6-97d1-4d96-9659-be25196f3228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853e254d4efd48ada7cd9963c7be9321'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.387063 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1100/2237] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:27 2022] DELETE /identity/v3/users/853e254d4efd48ada7cd9963c7be9321 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:27.651328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:27.652795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:27.666328 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5643f-0847-4333-b9e4-8ce836630862 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:27.713588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:27.724956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:27.725616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:27.725893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:27.726165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] PATH_INFO: `/v3/projects/f6ec7fc24b934d599380c122eaf8ae58` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:27.727295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6ec7fc24b934d599380c122eaf8ae58)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:27.729270 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:27.774870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6ec7fc24b934d599380c122eaf8ae58', 'request_id': 'req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.782238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6ec7fc24b934d599380c122eaf8ae58', 'request_id': 'req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.788380 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6ec7fc24b934d599380c122eaf8ae58', 'request_id': 'req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.798852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] The token cache is being invalidate because project f6ec7fc24b934d599380c122eaf8ae58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:27.799133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.799830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9f18fd02-c652-4b12-9e82-fc81fc0c98ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:27.802515 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1138/2238] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:27 2022] DELETE /identity/v3/projects/f6ec7fc24b934d599380c122eaf8ae58 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:27.808780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11d6b-0896-410e-888d-b70950f1ac8b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:27.809258 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11d6b-0896-410e-888d-b70950f1ac8b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:27.809386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11d6b-0896-410e-888d-b70950f1ac8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:27.831328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fbe11d6b-0896-410e-888d-b70950f1ac8b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:27.832533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fbe11d6b-0896-410e-888d-b70950f1ac8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:27.865180 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:27.893884 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1101/2239] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:27.901683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b06f8-dedb-4380-905f-3bce52ba3aed None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:27.901966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b06f8-dedb-4380-905f-3bce52ba3aed None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:27.902260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b06f8-dedb-4380-905f-3bce52ba3aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:27.922137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-212b06f8-dedb-4380-905f-3bce52ba3aed None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:27.923373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-212b06f8-dedb-4380-905f-3bce52ba3aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:27.931255 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1139/2240] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:27.935418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe11d6b-0896-410e-888d-b70950f1ac8b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:27.945070 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe11d6b-0896-410e-888d-b70950f1ac8b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:27.949710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:27.950811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:27.951086 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:27.951353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:27.951992 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:27.952291 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:27.953888 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:27.965072 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1102/2241] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:04:27.969716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b06f8-dedb-4380-905f-3bce52ba3aed None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:27.978858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b06f8-dedb-4380-905f-3bce52ba3aed None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:27.983462 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:27.984051 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:27.984360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:27.984634 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:27.985257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:27.987126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:28.014729 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1140/2242] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:27 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:28.018769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:28.028641 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326a5d2-270d-4636-938b-b84f8e44a94e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:28.083490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:28.084109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:28.084415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:28.084690 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:28.085346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:28.087241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:28.108219 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1103/2243] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:28 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:28.112278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:28.122942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e3f60-e04a-4b6f-b673-9e2d79fc62c5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:28.127447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:28.128040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:28.128610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:28.128885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:28.129519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:28.131092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:28.141711 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1141/2244] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:28.145357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:28.155160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300772cf-686e-4fd4-abe2-e8c4918f7885 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:28.159707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b21d5-14b0-47a5-9ee9-f1d058fb6e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:28.160319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b21d5-14b0-47a5-9ee9-f1d058fb6e6b admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:28.160604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b21d5-14b0-47a5-9ee9-f1d058fb6e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:28.160876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b21d5-14b0-47a5-9ee9-f1d058fb6e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:28.161509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b21d5-14b0-47a5-9ee9-f1d058fb6e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:28.163102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b21d5-14b0-47a5-9ee9-f1d058fb6e6b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:28.173437 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1104/2245] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:28.177307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:28.187446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e00b5-9870-4ac4-b3b2-b2bfbd112a68 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:28.191964 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:28.192631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:28.192902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:28.193168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] PATH_INFO: `/v3/projects/9fa71cdcda63429584e83f50f3544f3f/users/e1a72d7ff20440bfbb2fd4283022f5c4/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:28.208045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa71cdcda63429584e83f50f3544f3f, user_id=e1a72d7ff20440bfbb2fd4283022f5c4, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:28.210688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:28.223183 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1142/2246] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:28 2022] PUT /identity/v3/projects/9fa71cdcda63429584e83f50f3544f3f/users/e1a72d7ff20440bfbb2fd4283022f5c4/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:28.234816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:28.235102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:28.235377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:28.264884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] MFA Rules not processed for user `e1a72d7ff20440bfbb2fd4283022f5c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:28.266083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:28.328163 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1105/2247] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:28.339950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:28.342789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:28.397772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032b258-7e0e-429d-82c8-42e1f8850fe1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:28.401316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:28.401908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:28.402180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:28.402452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:28.410977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:28.412422 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:28.421223 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1143/2248] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:04:38.325231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=82,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:38.417167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:47.701998 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:47.703371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:47.758542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e0133b-17fd-42a0-b7b7-473ddd1ae5c9 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:47.762084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f227c-04dd-4563-89c5-34a6e2275e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:47.762681 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f227c-04dd-4563-89c5-34a6e2275e0a service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:47.762957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f227c-04dd-4563-89c5-34a6e2275e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:47.763225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f227c-04dd-4563-89c5-34a6e2275e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:47.764916 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-603f227c-04dd-4563-89c5-34a6e2275e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:47.831465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f227c-04dd-4563-89c5-34a6e2275e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:47.832925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603f227c-04dd-4563-89c5-34a6e2275e0a service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:47.842515 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1106/2249] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:48.154644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:48.165812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09aae1c-ed20-44ea-8617-565b3480f7d2 service glance] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:48.170291 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:48.170899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:48.171171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:48.171436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] PATH_INFO: `/v3/users/e1a72d7ff20440bfbb2fd4283022f5c4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:48.172581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a72d7ff20440bfbb2fd4283022f5c4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:48.175060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:48.223939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a72d7ff20440bfbb2fd4283022f5c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.231253 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a72d7ff20440bfbb2fd4283022f5c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.239600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a72d7ff20440bfbb2fd4283022f5c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.251861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a72d7ff20440bfbb2fd4283022f5c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.252541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a72d7ff20440bfbb2fd4283022f5c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.258516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1144/2250] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:48 2022] DELETE /identity/v3/users/e1a72d7ff20440bfbb2fd4283022f5c4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:48.263066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:48.263354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:48.263626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:48.283457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:48.284825 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:48.345029 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1107/2251] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:48.354668 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:48.356121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:48.412961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8566e456-3607-4006-b9c6-8dff099a2f46 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:48.416550 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd4d7a-ce08-4b33-9c55-2c08072e55e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:48.417160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4d7a-ce08-4b33-9c55-2c08072e55e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:48.417436 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4d7a-ce08-4b33-9c55-2c08072e55e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:48.417706 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd4d7a-ce08-4b33-9c55-2c08072e55e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:48.427375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd4d7a-ce08-4b33-9c55-2c08072e55e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:48.428847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cd4d7a-ce08-4b33-9c55-2c08072e55e5 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:48.438468 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1145/2252] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:48.723130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:48.731067 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:48.745873 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c5f37-4ff0-4aaa-ae20-c25b311fa35a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:48.751419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:48.752973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:48.753551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:48.753828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] PATH_INFO: `/v3/projects/9fa71cdcda63429584e83f50f3544f3f` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:48.755151 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa71cdcda63429584e83f50f3544f3f)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:48.757602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:48.807271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:48.818782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa71cdcda63429584e83f50f3544f3f', 'request_id': 'req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.834326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa71cdcda63429584e83f50f3544f3f', 'request_id': 'req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.841257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa71cdcda63429584e83f50f3544f3f', 'request_id': 'req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.853671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] The token cache is being invalidate because project 9fa71cdcda63429584e83f50f3544f3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:48.853994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.854769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-b70a8c11-5ee8-4d55-baaa-bb42b51ac8cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:48.856563 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1108/2253] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:48 2022] DELETE /identity/v3/projects/9fa71cdcda63429584e83f50f3544f3f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:48.861873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063a39-700f-4b83-a297-ef8fafcd0fbd None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:48.862155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063a39-700f-4b83-a297-ef8fafcd0fbd None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:48.862426 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9063a39-700f-4b83-a297-ef8fafcd0fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:48.883447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e9063a39-700f-4b83-a297-ef8fafcd0fbd None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:48.884684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e9063a39-700f-4b83-a297-ef8fafcd0fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:48.947837 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1146/2254] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:48.955741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb895c-dc14-4ebf-aff3-8f1576a7b64e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:48.956022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb895c-dc14-4ebf-aff3-8f1576a7b64e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:48.956289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb895c-dc14-4ebf-aff3-8f1576a7b64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:48.977281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a8cb895c-dc14-4ebf-aff3-8f1576a7b64e None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:48.978406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a8cb895c-dc14-4ebf-aff3-8f1576a7b64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:48.990221 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1109/2255] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:48.992855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063a39-700f-4b83-a297-ef8fafcd0fbd None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:49.003920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9063a39-700f-4b83-a297-ef8fafcd0fbd None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:49.009588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:49.010233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.010510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.010780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.011428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:49.011729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:49.013356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:49.026133 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1147/2256] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:04:49.030571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb895c-dc14-4ebf-aff3-8f1576a7b64e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:49.042247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb895c-dc14-4ebf-aff3-8f1576a7b64e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:49.049707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:49.050931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.051511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.051950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.052796 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:49.055917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:49.093190 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1110/2257] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:49 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:49.094646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:49.100203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:49.106024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7612cd8f-5a76-4929-baeb-9cc992afacb7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:49.176584 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:49.177278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.177644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.178164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.178705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:49.181246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:49.205729 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1148/2258] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:49 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:49.209655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:49.220874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeea02d-1f7f-467f-8edb-8236ea609349 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:49.227272 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:49.227771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.227987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.228288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.229612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:49.230575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:49.241841 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1111/2259] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:49.245777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:49.256934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92070c-ad02-4216-bd86-a44ac0b2223c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:49.262951 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfcdd7a-a639-461c-b7e3-c66e4579715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:49.264712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdd7a-a639-461c-b7e3-c66e4579715a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.265001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdd7a-a639-461c-b7e3-c66e4579715a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.265284 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfcdd7a-a639-461c-b7e3-c66e4579715a admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.265935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcdd7a-a639-461c-b7e3-c66e4579715a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:49.267683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfcdd7a-a639-461c-b7e3-c66e4579715a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:49.282495 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1149/2260] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:49.283638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:49.293495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b10780-24fb-445b-9515-b1966227f517 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:49.298627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:49.299288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.299571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.299845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] PATH_INFO: `/v3/projects/6e35738c072847f6915e748b337dc502/users/7a297281836b4b0fb6f3067653ad5d5a/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.323511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e35738c072847f6915e748b337dc502, user_id=7a297281836b4b0fb6f3067653ad5d5a, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:49.326347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:49.343930 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1112/2261] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:49 2022] PUT /identity/v3/projects/6e35738c072847f6915e748b337dc502/users/7a297281836b4b0fb6f3067653ad5d5a/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:49.355092 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.355547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.355816 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.388469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] MFA Rules not processed for user `7a297281836b4b0fb6f3067653ad5d5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:49.389704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:49.467679 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1150/2262] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:49.480438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:49.482058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:49.555257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3120b-1b91-4d1c-9887-67714e5c707e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:49.559607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:49.560580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:49.560966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:49.561451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:49.572205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:49.573729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:49.584434 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1113/2263] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:51.167338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:51.169570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:51.250929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04806831-166a-4bf3-a71d-79661224d9a8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:51.256277 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:51.256935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.257213 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.257487 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] PATH_INFO: `/v3/users/959991a17c69459dbd665c2ae20095ae` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.258625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=959991a17c69459dbd665c2ae20095ae)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:51.260605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:51.273345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:51.308358 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '959991a17c69459dbd665c2ae20095ae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.315599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '959991a17c69459dbd665c2ae20095ae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.323166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '959991a17c69459dbd665c2ae20095ae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.335439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '959991a17c69459dbd665c2ae20095ae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.336039 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-4e8ed516-5ebb-4a93-ba13-acef614bec7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '959991a17c69459dbd665c2ae20095ae'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.342110 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1151/2264] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:51 2022] DELETE /identity/v3/users/959991a17c69459dbd665c2ae20095ae => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:51.508890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:51.523646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b61c5-93b2-4c02-bb9a-7d5a1a460a45 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:51.579806 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:51.580541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.580821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.581091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] PATH_INFO: `/v3/projects/f639829059494fd49c12d49d52134692` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.582195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f639829059494fd49c12d49d52134692)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:51.584083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:51.600477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:51.632064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f639829059494fd49c12d49d52134692', 'request_id': 'req-10c596de-ddda-47a4-abc7-8f392c66fc92'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.639689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f639829059494fd49c12d49d52134692', 'request_id': 'req-10c596de-ddda-47a4-abc7-8f392c66fc92'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.645686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f639829059494fd49c12d49d52134692', 'request_id': 'req-10c596de-ddda-47a4-abc7-8f392c66fc92'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.656463 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] The token cache is being invalidate because project f639829059494fd49c12d49d52134692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:51.656769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.657936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-10c596de-ddda-47a4-abc7-8f392c66fc92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:51.659391 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1114/2265] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:51 2022] DELETE /identity/v3/projects/f639829059494fd49c12d49d52134692 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:51.664178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb841f-817d-4be6-b07c-e030c265fdfe None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.664509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb841f-817d-4be6-b07c-e030c265fdfe None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.664781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb841f-817d-4be6-b07c-e030c265fdfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.683679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5ddb841f-817d-4be6-b07c-e030c265fdfe None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:51.684898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5ddb841f-817d-4be6-b07c-e030c265fdfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:51.741717 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1152/2266] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:51.749524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cf4c5-70cc-4390-8a8a-ad9dbf970950 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.749805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cf4c5-70cc-4390-8a8a-ad9dbf970950 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.750075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cf4c5-70cc-4390-8a8a-ad9dbf970950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.770195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-fc3cf4c5-70cc-4390-8a8a-ad9dbf970950 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:51.771410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fc3cf4c5-70cc-4390-8a8a-ad9dbf970950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:51.779303 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1115/2267] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:51.783875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb841f-817d-4be6-b07c-e030c265fdfe None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:51.794231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb841f-817d-4be6-b07c-e030c265fdfe None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:51.798876 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:51.799495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.799767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.800035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.800716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:51.801019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:51.802607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:51.814405 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1153/2268] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:04:51.818732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cf4c5-70cc-4390-8a8a-ad9dbf970950 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:51.828008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cf4c5-70cc-4390-8a8a-ad9dbf970950 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:51.833186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:51.833776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.834049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.834312 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.834933 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:51.836833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:51.863509 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1116/2269] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:51 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:04:51.867577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:51.878070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f21a979-2a74-45aa-9a06-2212462925c5 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:51.932595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:51.933280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.933565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.933839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.934458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:51.936394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:51.957502 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1154/2270] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:51 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:51.961608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:51.971644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df870f84-a24f-4562-9909-3ca4ca5eec88 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:51.976333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:51.976926 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:51.977199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:51.977464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:51.978103 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:51.979711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:51.989494 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1117/2271] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:51.993119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.004066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99931ba-4143-4776-b4bd-39b1a28f9192 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.008766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c76ea-bf08-43ef-86e5-b6448740e98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.009356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c76ea-bf08-43ef-86e5-b6448740e98d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.009631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c76ea-bf08-43ef-86e5-b6448740e98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.009897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c76ea-bf08-43ef-86e5-b6448740e98d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.010529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c76ea-bf08-43ef-86e5-b6448740e98d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.012103 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c76ea-bf08-43ef-86e5-b6448740e98d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.013460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:52.029217 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1155/2272] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:52.030420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.046729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ecee30-9fa5-4a58-be05-5d724ae2d7e9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.051585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.052243 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.052545 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.052944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] PATH_INFO: `/v3/projects/295197855ff740cbaafaabcbecdaa59c/users/b9b5b6ac7ca943efb1f2f30a0a7e3f1d/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.069066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295197855ff740cbaafaabcbecdaa59c, user_id=b9b5b6ac7ca943efb1f2f30a0a7e3f1d, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.071785 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.085610 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1118/2273] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:52 2022] PUT /identity/v3/projects/295197855ff740cbaafaabcbecdaa59c/users/b9b5b6ac7ca943efb1f2f30a0a7e3f1d/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:52.095817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b0828-16f4-4a18-b169-9f6d42c20736 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.096107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b0828-16f4-4a18-b169-9f6d42c20736 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.096423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b0828-16f4-4a18-b169-9f6d42c20736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.124168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-2c6b0828-16f4-4a18-b169-9f6d42c20736 None None] MFA Rules not processed for user `b9b5b6ac7ca943efb1f2f30a0a7e3f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:52.125456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2c6b0828-16f4-4a18-b169-9f6d42c20736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:52.189670 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1156/2274] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:52.194111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.255218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fe8e2-984a-4940-b415-ae3c04bfbdc8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.259793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.260420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.260700 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.260970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.261605 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.263493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.292396 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1119/2275] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:52 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:52.295029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b0828-16f4-4a18-b169-9f6d42c20736 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.299884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.301567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:52.308583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b0828-16f4-4a18-b169-9f6d42c20736 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.334816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:52.374077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.374893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.375117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.375391 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.376691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.379082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.382062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e11cac-2b6e-44ad-a3d0-fa1601f15b03 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.386767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.387425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.387707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.387972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] PATH_INFO: `/v3/users/7a297281836b4b0fb6f3067653ad5d5a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.389133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a297281836b4b0fb6f3067653ad5d5a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.391106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.401556 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1157/2276] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:52 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:52.405440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.415420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736a3dd-dbf0-4c04-bfed-384fe47e3dcf admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.420633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.421249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.421512 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.421825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.423056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.424812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.435568 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1158/2277] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:52.439987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a297281836b4b0fb6f3067653ad5d5a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.440263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.440760 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a297281836b4b0fb6f3067653ad5d5a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.442086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:52.447741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a297281836b4b0fb6f3067653ad5d5a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.456037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af13fadb-f7e9-485c-aae7-bbace3617fdd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.459604 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a297281836b4b0fb6f3067653ad5d5a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.460277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-06c8459c-d9cc-4780-8e9d-6a08309d9959 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a297281836b4b0fb6f3067653ad5d5a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.460767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.461421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.461680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.461950 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] PATH_INFO: `/v3/projects/7598e3ac98a043ae98060d9f1e743a76/users/f4e54fa236a74565b1bded848ef43f61/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.474011 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1120/2278] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:52 2022] DELETE /identity/v3/users/7a297281836b4b0fb6f3067653ad5d5a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:52.477873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7598e3ac98a043ae98060d9f1e743a76, user_id=f4e54fa236a74565b1bded848ef43f61, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.478326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.478617 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.478899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.480699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.494628 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1159/2279] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:52 2022] PUT /identity/v3/projects/7598e3ac98a043ae98060d9f1e743a76/users/f4e54fa236a74565b1bded848ef43f61/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:52.498377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.499217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:52.500452 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:52.543529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:52.561295 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1121/2280] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:52.565323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded6309-c790-456e-a7d9-30265976863a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.570074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.570685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.570957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.571225 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.571863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.573300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.573460 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.574653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:52.587311 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1160/2281] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:52.593680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.603573 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f245cc-04cf-40c0-82e2-04840863e1c1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.608131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079107f-e6bc-45d2-ab9a-eaf754621bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.608873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079107f-e6bc-45d2-ab9a-eaf754621bea admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.609148 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079107f-e6bc-45d2-ab9a-eaf754621bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.609418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079107f-e6bc-45d2-ab9a-eaf754621bea admin admin] PATH_INFO: `/v3/projects/7598e3ac98a043ae98060d9f1e743a76/users/f4e54fa236a74565b1bded848ef43f61/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.611848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079107f-e6bc-45d2-ab9a-eaf754621bea admin admin] RBAC: Authorizing `identity:create_grant(project_id=7598e3ac98a043ae98060d9f1e743a76, user_id=f4e54fa236a74565b1bded848ef43f61, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.614492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079107f-e6bc-45d2-ab9a-eaf754621bea admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.622382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:52.628559 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1161/2282] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:52 2022] PUT /identity/v3/projects/7598e3ac98a043ae98060d9f1e743a76/users/f4e54fa236a74565b1bded848ef43f61/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:52.637435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf74a91-bc39-4025-9a64-5ca53e5c02d1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.639043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.639043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.639238 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.668917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] MFA Rules not processed for user `f4e54fa236a74565b1bded848ef43f61`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:52.670190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:52.686425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.687029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.687307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.687662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.744718 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1162/2283] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:52.750696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.752129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.756692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.758076 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:52.762715 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1122/2284] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:52.815843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f595b7d-f409-4660-961f-a83d21ec206b None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.820031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aeab5f-78d0-4c2f-aefa-eddba57c444c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.820833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeab5f-78d0-4c2f-aefa-eddba57c444c service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.821111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeab5f-78d0-4c2f-aefa-eddba57c444c service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.821448 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aeab5f-78d0-4c2f-aefa-eddba57c444c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.831195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aeab5f-78d0-4c2f-aefa-eddba57c444c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.832796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68aeab5f-78d0-4c2f-aefa-eddba57c444c service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.843352 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1163/2285] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:52 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:04:52.873136 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:52.883319 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b6bf3-e220-4ab8-aabe-2a704f963075 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:52.887943 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:52.888609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.888896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.889172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] PATH_INFO: `/v3/users/b9b5b6ac7ca943efb1f2f30a0a7e3f1d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:52.890334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b5b6ac7ca943efb1f2f30a0a7e3f1d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:52.892273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:52.946242 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b5b6ac7ca943efb1f2f30a0a7e3f1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.946902 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b5b6ac7ca943efb1f2f30a0a7e3f1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.954719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b5b6ac7ca943efb1f2f30a0a7e3f1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.967235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b5b6ac7ca943efb1f2f30a0a7e3f1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.967934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b5b6ac7ca943efb1f2f30a0a7e3f1d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:52.981158 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1123/2286] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:52 2022] DELETE /identity/v3/users/b9b5b6ac7ca943efb1f2f30a0a7e3f1d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:52.986180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:52.986574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:52.986865 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.007193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:53.008433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.044077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.045628 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.057118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:53.075197 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1164/2287] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:53.085075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.086435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.090908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:53.134570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bd6b9e-7b39-4b33-a6e1-88b8ae776ffd admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.139185 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.139822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.140105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.140403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] PATH_INFO: `/v3/projects/6e35738c072847f6915e748b337dc502` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.141611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e35738c072847f6915e748b337dc502)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.143501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.145232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2eb43c-9ca7-497b-95f4-352893cd9f90 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.148875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a95403-6228-4568-9c1b-0562c451ced1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.149485 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95403-6228-4568-9c1b-0562c451ced1 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.149763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95403-6228-4568-9c1b-0562c451ced1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.150032 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a95403-6228-4568-9c1b-0562c451ced1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.159420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95403-6228-4568-9c1b-0562c451ced1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.161077 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a95403-6228-4568-9c1b-0562c451ced1 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.170602 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1165/2288] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:04:53.192849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e35738c072847f6915e748b337dc502', 'request_id': 'req-0267d4e5-834d-4241-b072-56b754125c66'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.200812 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e35738c072847f6915e748b337dc502', 'request_id': 'req-0267d4e5-834d-4241-b072-56b754125c66'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.207081 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e35738c072847f6915e748b337dc502', 'request_id': 'req-0267d4e5-834d-4241-b072-56b754125c66'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.218694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] The token cache is being invalidate because project 6e35738c072847f6915e748b337dc502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:53.219368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.220178 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-0267d4e5-834d-4241-b072-56b754125c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.221969 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1124/2289] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:53 2022] DELETE /identity/v3/projects/6e35738c072847f6915e748b337dc502 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:53.226972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08f03e-e0a7-4391-b00e-645aa06e419c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.227470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08f03e-e0a7-4391-b00e-645aa06e419c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.227588 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb08f03e-e0a7-4391-b00e-645aa06e419c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.248075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-eb08f03e-e0a7-4391-b00e-645aa06e419c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:53.249317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eb08f03e-e0a7-4391-b00e-645aa06e419c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.310314 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1166/2290] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:53.319931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6649ff9-e0d3-4e20-b4bf-12354797931b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.320186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6649ff9-e0d3-4e20-b4bf-12354797931b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.320728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6649ff9-e0d3-4e20-b4bf-12354797931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.345027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-c6649ff9-e0d3-4e20-b4bf-12354797931b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:53.346221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c6649ff9-e0d3-4e20-b4bf-12354797931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.354229 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1125/2291] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:53.358452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08f03e-e0a7-4391-b00e-645aa06e419c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.368497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb08f03e-e0a7-4391-b00e-645aa06e419c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.373281 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.373888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.374168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.374440 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.375098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:53.375401 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.377743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.390165 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1167/2292] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:04:53.395092 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6649ff9-e0d3-4e20-b4bf-12354797931b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.404111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6649ff9-e0d3-4e20-b4bf-12354797931b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.408805 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.409426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.409696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.409962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.410581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.412475 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.442527 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1126/2293] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:53 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:53.446942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.447744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.450311 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.466037 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e374c48-5ceb-408e-9083-9a91542554bf admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.509871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bcaf-a648-4ecf-8c8e-3209fd256ddb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.514693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.515296 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.515609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.515885 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.517143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.519058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.526540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.527180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.527458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.527743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] PATH_INFO: `/v3/projects/295197855ff740cbaafaabcbecdaa59c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.528956 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=295197855ff740cbaafaabcbecdaa59c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.530850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.540760 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1127/2294] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:53 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:53.545012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.554541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1aa33d-6158-4a69-a5d5-dfb5b4d6b942 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.555809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:53.560640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.561607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.562213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.562298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.563570 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.565723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.579461 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1128/2295] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:53.580393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '295197855ff740cbaafaabcbecdaa59c', 'request_id': 'req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.580870 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.585667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:53.588174 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '295197855ff740cbaafaabcbecdaa59c', 'request_id': 'req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.592865 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9fd321-c6c2-4b02-b933-8306fef3cf92 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.596439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '295197855ff740cbaafaabcbecdaa59c', 'request_id': 'req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.609389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] The token cache is being invalidate because project 295197855ff740cbaafaabcbecdaa59c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:53.610083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.611061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.613183 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1168/2296] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:53 2022] DELETE /identity/v3/projects/295197855ff740cbaafaabcbecdaa59c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:53.618696 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.620457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.659060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.659591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.659872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.660142 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.660846 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.662426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.672506 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1129/2297] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:53 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:04:53.676756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.678387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.686302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5fb0c1-c6d5-4af4-ae1a-e17e0d2e9359 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.690961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be8db62-15e6-4a54-a4d5-f7764aacc8e3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.691427 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.692055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.692356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.692635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] PATH_INFO: `/v3/users/f4e54fa236a74565b1bded848ef43f61` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.693751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4e54fa236a74565b1bded848ef43f61)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.695644 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.695828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ec8e5-1633-4413-8beb-af2951c712ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.696497 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec8e5-1633-4413-8beb-af2951c712ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.696835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec8e5-1633-4413-8beb-af2951c712ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.697289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ec8e5-1633-4413-8beb-af2951c712ea admin admin] PATH_INFO: `/v3/projects/b44d7361c05546b0bc525a9dfe170f23/users/4520c2c66d38454a890f81cb82545f56/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.713187 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ec8e5-1633-4413-8beb-af2951c712ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44d7361c05546b0bc525a9dfe170f23, user_id=4520c2c66d38454a890f81cb82545f56, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.715835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ec8e5-1633-4413-8beb-af2951c712ea admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:53.731053 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1130/2298] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:53 2022] PUT /identity/v3/projects/b44d7361c05546b0bc525a9dfe170f23/users/4520c2c66d38454a890f81cb82545f56/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:04:53.741996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.742283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.742564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.745869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4e54fa236a74565b1bded848ef43f61'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.754192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4e54fa236a74565b1bded848ef43f61'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.761713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4e54fa236a74565b1bded848ef43f61'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.771733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] MFA Rules not processed for user `4520c2c66d38454a890f81cb82545f56`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:53.772956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.774374 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4e54fa236a74565b1bded848ef43f61'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.775040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4e54fa236a74565b1bded848ef43f61'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:53.780872 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1169/2299] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:53 2022] DELETE /identity/v3/users/f4e54fa236a74565b1bded848ef43f61 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:53.848109 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1131/2300] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:53.855884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:53.857488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:53.894310 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:53.921825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa13804-9150-4884-b66d-b07d32aa782f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:53.929737 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1754281c-3fd3-4458-9f9d-274174067252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:53.930368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754281c-3fd3-4458-9f9d-274174067252 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:53.930647 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754281c-3fd3-4458-9f9d-274174067252 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:53.930917 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1754281c-3fd3-4458-9f9d-274174067252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:53.989398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754281c-3fd3-4458-9f9d-274174067252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:53.990870 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1754281c-3fd3-4458-9f9d-274174067252 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.000188 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1170/2301] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:54.058237 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.059653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:54.130684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6383bd3-a799-4811-a3e8-b1de208fcd3e None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.138921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:54.191607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.192245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.192556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.192828 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] PATH_INFO: `/v3/projects/7598e3ac98a043ae98060d9f1e743a76` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.194004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7598e3ac98a043ae98060d9f1e743a76)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.195925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.244280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7598e3ac98a043ae98060d9f1e743a76', 'request_id': 'req-f7518f8f-3c54-4d4d-9595-308ec3217709'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:54.252790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7598e3ac98a043ae98060d9f1e743a76', 'request_id': 'req-f7518f8f-3c54-4d4d-9595-308ec3217709'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:54.259874 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7598e3ac98a043ae98060d9f1e743a76', 'request_id': 'req-f7518f8f-3c54-4d4d-9595-308ec3217709'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:54.273339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] The token cache is being invalidate because project 7598e3ac98a043ae98060d9f1e743a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:54.273589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:54.274301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f7518f8f-3c54-4d4d-9595-308ec3217709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:54.276088 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1132/2302] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:54 2022] DELETE /identity/v3/projects/7598e3ac98a043ae98060d9f1e743a76 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:54.280993 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132996-b57c-489a-a050-b4a71600b022 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.281494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132996-b57c-489a-a050-b4a71600b022 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.281583 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4132996-b57c-489a-a050-b4a71600b022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.301186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-b4132996-b57c-489a-a050-b4a71600b022 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:54.302423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b4132996-b57c-489a-a050-b4a71600b022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:54.364848 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1171/2303] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:54.372717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c554a01-0ad2-4a69-8328-32d24f42a091 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.373000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c554a01-0ad2-4a69-8328-32d24f42a091 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.373292 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c554a01-0ad2-4a69-8328-32d24f42a091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.393944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-8c554a01-0ad2-4a69-8328-32d24f42a091 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:54.395138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8c554a01-0ad2-4a69-8328-32d24f42a091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:54.402869 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1133/2304] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:54.407222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4132996-b57c-489a-a050-b4a71600b022 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.416857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4132996-b57c-489a-a050-b4a71600b022 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.421413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241393-1138-4df2-b486-98121138a425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.422011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241393-1138-4df2-b486-98121138a425 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.422381 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241393-1138-4df2-b486-98121138a425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.422599 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77241393-1138-4df2-b486-98121138a425 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.423245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77241393-1138-4df2-b486-98121138a425 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:04:54.423548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77241393-1138-4df2-b486-98121138a425 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.425165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77241393-1138-4df2-b486-98121138a425 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.436538 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1172/2305] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:04:54.441331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c554a01-0ad2-4a69-8328-32d24f42a091 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.450472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c554a01-0ad2-4a69-8328-32d24f42a091 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.455635 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.456229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.456542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.456816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.457477 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.459370 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.485857 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1134/2306] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:54 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:54.489965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241393-1138-4df2-b486-98121138a425 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.499916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77241393-1138-4df2-b486-98121138a425 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.556157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.556786 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.557067 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.557354 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.557972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.559885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.580879 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1173/2307] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:54 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:54.584940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.596163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2595a8a-54f3-45e1-ae3f-fc24062e8468 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.600792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.601390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.601698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.601972 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.602609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.604192 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.614713 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1135/2308] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:54.618594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.623648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:54.630160 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b510300-b066-4876-94e9-45e5c64f29cd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.635372 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b019ea48-9483-4187-b962-d129b657e13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.635969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019ea48-9483-4187-b962-d129b657e13c admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.636250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019ea48-9483-4187-b962-d129b657e13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.636554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019ea48-9483-4187-b962-d129b657e13c admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.637199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019ea48-9483-4187-b962-d129b657e13c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.638777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b019ea48-9483-4187-b962-d129b657e13c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.649325 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1174/2309] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:54.653279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.662979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881852b0-862e-40cb-a2b1-0c712e4b92ae admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.667433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.668068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.668372 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.668642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] PATH_INFO: `/v3/projects/874913d4f3c2415792c3a07a772c1c4e/users/f4021fdebe5949ad8f99016ebe61f5b0/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.683255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=874913d4f3c2415792c3a07a772c1c4e, user_id=f4021fdebe5949ad8f99016ebe61f5b0, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.685908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.698951 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1136/2310] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:54 2022] PUT /identity/v3/projects/874913d4f3c2415792c3a07a772c1c4e/users/f4021fdebe5949ad8f99016ebe61f5b0/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:54.709133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fe571-c6af-4d9a-9430-b307729378db None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.709420 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fe571-c6af-4d9a-9430-b307729378db None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.709718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fe571-c6af-4d9a-9430-b307729378db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.738170 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-637fe571-c6af-4d9a-9430-b307729378db None None] MFA Rules not processed for user `f4021fdebe5949ad8f99016ebe61f5b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:54.739377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-637fe571-c6af-4d9a-9430-b307729378db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:54.806786 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1175/2311] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:54.811395 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.853082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:54.876146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc878d51-29b3-4360-bcd7-3d9b861dab83 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.881147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.881743 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.882016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.882281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.882908 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.884811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:54.910350 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1137/2312] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:04:54 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:54.914531 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fe571-c6af-4d9a-9430-b307729378db None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:54.925555 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fe571-c6af-4d9a-9430-b307729378db None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:54.978689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:54.979206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:54.979479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:54.979748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:54.980398 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:54.982278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:55.003640 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1176/2313] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:04:54 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:04:55.007606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:55.017580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd9da3-26e1-4eaf-bb45-4c01884892c8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:55.022057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:55.022855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.023134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.023476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.024107 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:55.025788 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:55.037036 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1138/2314] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:55.041136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:55.051362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708678b0-f183-4151-8c21-098f2d9c1981 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:55.056034 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:55.056699 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.056976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.057247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] PATH_INFO: `/v3/projects/dea0ac109feb4fceb946726c3d8356a5/users/4b37fc92d3c74aa89da209444b0b2e5e/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.071468 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea0ac109feb4fceb946726c3d8356a5, user_id=4b37fc92d3c74aa89da209444b0b2e5e, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:55.074134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:55.086385 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1177/2315] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:55 2022] PUT /identity/v3/projects/dea0ac109feb4fceb946726c3d8356a5/users/4b37fc92d3c74aa89da209444b0b2e5e/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:55.090279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:55.100771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fba1bb8-bf14-4f7b-a00a-49a4a6740ceb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:55.155533 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e166ce-e878-4e11-b9f8-1a4c4852114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:55.156134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e166ce-e878-4e11-b9f8-1a4c4852114f admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.156445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e166ce-e878-4e11-b9f8-1a4c4852114f admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.157274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e166ce-e878-4e11-b9f8-1a4c4852114f admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.157909 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e166ce-e878-4e11-b9f8-1a4c4852114f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:55.159489 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e166ce-e878-4e11-b9f8-1a4c4852114f admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:55.168934 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1139/2316] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:04:55.173618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:55.183591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc1603-e147-4156-89df-6ffe09f97446 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:55.188222 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:55.188896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.189178 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.189454 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] PATH_INFO: `/v3/projects/dea0ac109feb4fceb946726c3d8356a5/users/4b37fc92d3c74aa89da209444b0b2e5e/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.191355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dea0ac109feb4fceb946726c3d8356a5, user_id=4b37fc92d3c74aa89da209444b0b2e5e, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:55.194031 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:55.203978 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1178/2317] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:04:55 2022] PUT /identity/v3/projects/dea0ac109feb4fceb946726c3d8356a5/users/4b37fc92d3c74aa89da209444b0b2e5e/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:55.215033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903023bf-f3ed-45ee-baa6-66867230ccb3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.215318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903023bf-f3ed-45ee-baa6-66867230ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.215591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903023bf-f3ed-45ee-baa6-66867230ccb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.243942 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-903023bf-f3ed-45ee-baa6-66867230ccb3 None None] MFA Rules not processed for user `4b37fc92d3c74aa89da209444b0b2e5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:55.245087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-903023bf-f3ed-45ee-baa6-66867230ccb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:55.311133 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1140/2318] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:55.322424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:55.324205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:55.363182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:55.377297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0860c6-219d-4c67-9eb9-560a56dd5008 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.377581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0860c6-219d-4c67-9eb9-560a56dd5008 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.377853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0860c6-219d-4c67-9eb9-560a56dd5008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.388439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154b1a6c-7708-4d21-b7ef-dd613407e7b2 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:55.392848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:55.393458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.393739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.394012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.397428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-0e0860c6-219d-4c67-9eb9-560a56dd5008 None None] MFA Rules not processed for user `4520c2c66d38454a890f81cb82545f56`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:55.398624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-0e0860c6-219d-4c67-9eb9-560a56dd5008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:55.403104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:55.404560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:55.414862 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1179/2319] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:55.428565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:55.458574 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1141/2320] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:04:55.519569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:55.528164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8889ee-6e8b-435c-8c38-1d1592122d9a service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:55.531749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32fdce6-3656-498f-a39c-53169bc93dee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:55.532360 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fdce6-3656-498f-a39c-53169bc93dee service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:55.532660 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fdce6-3656-498f-a39c-53169bc93dee service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:55.532929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32fdce6-3656-498f-a39c-53169bc93dee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:55.590324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fdce6-3656-498f-a39c-53169bc93dee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:55.591808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32fdce6-3656-498f-a39c-53169bc93dee service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:55.601688 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1180/2321] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:55 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:04:58.239179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0860c6-219d-4c67-9eb9-560a56dd5008 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:58.317281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0860c6-219d-4c67-9eb9-560a56dd5008 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:58.322069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:58.322768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:58.323109 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:58.323443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] PATH_INFO: `/v3/users/f4021fdebe5949ad8f99016ebe61f5b0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:58.324729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4021fdebe5949ad8f99016ebe61f5b0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:58.326670 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:58.381734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4021fdebe5949ad8f99016ebe61f5b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:58.382525 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4021fdebe5949ad8f99016ebe61f5b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:58.390928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4021fdebe5949ad8f99016ebe61f5b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:58.404961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4021fdebe5949ad8f99016ebe61f5b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:58.405811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4021fdebe5949ad8f99016ebe61f5b0'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:58.424206 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1142/2322] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:58 2022] DELETE /identity/v3/users/f4021fdebe5949ad8f99016ebe61f5b0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:58.426151 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:58.426612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:58.426733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:58.450568 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:04:58.452091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:58.524246 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1181/2323] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:04:58.539229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:58.541245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:58.627511 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b90c9b-12ab-4855-b4ed-923a5828fed3 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:58.631465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:58.632078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:58.632388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:58.632662 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:58.642852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:58.644398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:58.650790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:58.661069 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1143/2324] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:04:59.305632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:59.307265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:04:59.330963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b43c1-a80a-46f7-8fab-b4c3630b702f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:59.337096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:59.337898 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:59.338383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:59.339306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] PATH_INFO: `/v3/projects/874913d4f3c2415792c3a07a772c1c4e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:59.340963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=874913d4f3c2415792c3a07a772c1c4e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:59.343102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:59.437424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '874913d4f3c2415792c3a07a772c1c4e', 'request_id': 'req-3e8d212d-e6d9-4b4b-a40d-de59907620a3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.454514 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '874913d4f3c2415792c3a07a772c1c4e', 'request_id': 'req-3e8d212d-e6d9-4b4b-a40d-de59907620a3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.463345 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '874913d4f3c2415792c3a07a772c1c4e', 'request_id': 'req-3e8d212d-e6d9-4b4b-a40d-de59907620a3'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.473274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:04:59.478783 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] The token cache is being invalidate because project 874913d4f3c2415792c3a07a772c1c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:04:59.479466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.480709 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.484150 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1182/2325] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:04:59 2022] DELETE /identity/v3/projects/874913d4f3c2415792c3a07a772c1c4e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:04:59.488181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:04:59.509592 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6634cb5b-c882-4d96-9dbd-50e04eca13ef service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:04:59.586000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:04:59.586642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:04:59.587215 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:04:59.587505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] PATH_INFO: `/v3/users/4b37fc92d3c74aa89da209444b0b2e5e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:04:59.588827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b37fc92d3c74aa89da209444b0b2e5e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:04:59.591084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:04:59.648934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b37fc92d3c74aa89da209444b0b2e5e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.649587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b37fc92d3c74aa89da209444b0b2e5e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.657932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b37fc92d3c74aa89da209444b0b2e5e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.671606 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b37fc92d3c74aa89da209444b0b2e5e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.673132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e352aca3-c804-4e04-b9f0-fff8f03cb218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b37fc92d3c74aa89da209444b0b2e5e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:04:59.690168 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1144/2326] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:04:59 2022] DELETE /identity/v3/users/4b37fc92d3c74aa89da209444b0b2e5e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:00.030227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.032397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:00.052837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d212d-e6d9-4b4b-a40d-de59907620a3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:00.123329 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:00.124250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.124787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.125998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] PATH_INFO: `/v3/projects/dea0ac109feb4fceb946726c3d8356a5` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.128144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dea0ac109feb4fceb946726c3d8356a5)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:00.131521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:00.211417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dea0ac109feb4fceb946726c3d8356a5', 'request_id': 'req-a969bfc5-fa9b-4c20-b685-712544ebb906'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:00.226241 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dea0ac109feb4fceb946726c3d8356a5', 'request_id': 'req-a969bfc5-fa9b-4c20-b685-712544ebb906'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:00.233563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dea0ac109feb4fceb946726c3d8356a5', 'request_id': 'req-a969bfc5-fa9b-4c20-b685-712544ebb906'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:00.253571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] The token cache is being invalidate because project dea0ac109feb4fceb946726c3d8356a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:00.253571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:00.254478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a969bfc5-fa9b-4c20-b685-712544ebb906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:00.257056 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1183/2327] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:00 2022] DELETE /identity/v3/projects/dea0ac109feb4fceb946726c3d8356a5 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:00.261940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba7d18-9f5c-43e5-93dc-95bdfcce5e01 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.262341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba7d18-9f5c-43e5-93dc-95bdfcce5e01 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.262713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba7d18-9f5c-43e5-93dc-95bdfcce5e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.306730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2cba7d18-9f5c-43e5-93dc-95bdfcce5e01 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:00.316016 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2cba7d18-9f5c-43e5-93dc-95bdfcce5e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:00.404428 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1145/2328] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:00.411638 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7f69-f7f0-4b28-81b7-af82fd074007 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.411921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7f69-f7f0-4b28-81b7-af82fd074007 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.412191 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde7f69-f7f0-4b28-81b7-af82fd074007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.435496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-efde7f69-f7f0-4b28-81b7-af82fd074007 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:00.436777 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-efde7f69-f7f0-4b28-81b7-af82fd074007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:00.446692 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1184/2329] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:00.450747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba7d18-9f5c-43e5-93dc-95bdfcce5e01 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.463581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba7d18-9f5c-43e5-93dc-95bdfcce5e01 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:00.468953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:00.469607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.469882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.470152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.470807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:00.471580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:00.472816 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:00.488222 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1146/2330] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:05:00.490653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde7f69-f7f0-4b28-81b7-af82fd074007 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.501554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde7f69-f7f0-4b28-81b7-af82fd074007 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:00.506757 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:00.507376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.507873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.507990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.508587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:00.510656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:00.541927 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1185/2331] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:00 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:00.546170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.560203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d66a6b-e431-4532-a408-866094cf55e8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:00.564759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:00.650417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:00.651038 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.651490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.651775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.652821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:00.655829 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:00.690874 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1147/2332] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:00 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:00.694681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.707795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4bf57a-1d84-4051-9c9a-54f2d66adbc4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:00.712869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:00.713483 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.714004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.714278 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.715107 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:00.716918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:00.731690 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1186/2333] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:00.732973 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.745744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7955f-7794-4f45-89d1-91b6659d715c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:00.751766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d715f-03ac-4e44-b577-3d3721d1ed5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:00.752410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d715f-03ac-4e44-b577-3d3721d1ed5c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.752762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d715f-03ac-4e44-b577-3d3721d1ed5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.752939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d715f-03ac-4e44-b577-3d3721d1ed5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.753836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d715f-03ac-4e44-b577-3d3721d1ed5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:00.755412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d715f-03ac-4e44-b577-3d3721d1ed5c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:00.770814 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1148/2334] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:00.772015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.783813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd802d0-7de6-427d-8926-2cf64fa0fd29 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:00.788661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:00.789322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.789601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.789875 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] PATH_INFO: `/v3/projects/bb9bd828b0814dcf9a487330efa76638/users/2720cc702fcc413dbb2edf75a6b9ef6d/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.806944 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9bd828b0814dcf9a487330efa76638, user_id=2720cc702fcc413dbb2edf75a6b9ef6d, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:00.809748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:00.824411 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1187/2335] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:00 2022] PUT /identity/v3/projects/bb9bd828b0814dcf9a487330efa76638/users/2720cc702fcc413dbb2edf75a6b9ef6d/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:00.835407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b215a3-66e6-45b7-8188-8e442a695ad2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:00.835792 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b215a3-66e6-45b7-8188-8e442a695ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:00.836164 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b215a3-66e6-45b7-8188-8e442a695ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:00.865072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-80b215a3-66e6-45b7-8188-8e442a695ad2 None None] MFA Rules not processed for user `2720cc702fcc413dbb2edf75a6b9ef6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:00.866203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-80b215a3-66e6-45b7-8188-8e442a695ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:00.932514 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1149/2336] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:00.936925 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:00.964878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:01.005784 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecced0b-6e5c-462c-9417-f50e59393f5a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.010924 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.011633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.012048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.012452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.013086 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.015136 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.042635 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1188/2337] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:00 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:01.046860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b215a3-66e6-45b7-8188-8e442a695ad2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.058099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b215a3-66e6-45b7-8188-8e442a695ad2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.119914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.120467 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.120745 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.121012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.121646 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.123535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.151921 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1150/2338] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:01 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:01.157169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.167561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3618a55-5bdf-416a-9ec5-8d9168fc36d4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.172188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.172820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.173102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.173376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.174119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.175714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.186408 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1189/2339] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:01.190996 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.198023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:01.202895 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e30266-db98-4e3c-a65b-18031901e96e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.213338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601bad5-1fdb-4c06-9676-b0c245b34d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.213338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601bad5-1fdb-4c06-9676-b0c245b34d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.213338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601bad5-1fdb-4c06-9676-b0c245b34d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.213338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601bad5-1fdb-4c06-9676-b0c245b34d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.213715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601bad5-1fdb-4c06-9676-b0c245b34d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.216134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601bad5-1fdb-4c06-9676-b0c245b34d8a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.230796 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1151/2340] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:01.232068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.243562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662d6755-71ad-4118-9846-d90c80e8476d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.248377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.249029 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.249307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.249578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] PATH_INFO: `/v3/projects/93e229d777994471b526b6b65ba1c82e/users/4e16ec9271d34d1aa1e878fcd6a6ecc1/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.266563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] RBAC: Authorizing `identity:create_grant(project_id=93e229d777994471b526b6b65ba1c82e, user_id=4e16ec9271d34d1aa1e878fcd6a6ecc1, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.269307 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.282060 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1190/2341] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:01 2022] PUT /identity/v3/projects/93e229d777994471b526b6b65ba1c82e/users/4e16ec9271d34d1aa1e878fcd6a6ecc1/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:01.293578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c73ae5-1a73-4fe5-bb29-f8296c5109cb None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.293892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c73ae5-1a73-4fe5-bb29-f8296c5109cb None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.294188 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c73ae5-1a73-4fe5-bb29-f8296c5109cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.324353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-10c73ae5-1a73-4fe5-bb29-f8296c5109cb None None] MFA Rules not processed for user `4e16ec9271d34d1aa1e878fcd6a6ecc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:01.325587 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-10c73ae5-1a73-4fe5-bb29-f8296c5109cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:01.395385 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1152/2342] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:01.399828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.461569 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b06240e-bb53-4baa-b7fe-bdb548c9044a admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.466387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.466992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.467269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.467541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.468164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.470162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.501132 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1191/2343] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:01 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:01.502277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c73ae5-1a73-4fe5-bb29-f8296c5109cb None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.512927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c73ae5-1a73-4fe5-bb29-f8296c5109cb None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.568706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.569308 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.569583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.569854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.570468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.572374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.592941 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1153/2344] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:01 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:01.597399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.608060 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540ac78-5597-4b27-8d6b-19d2b2fc3201 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.613288 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.613879 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.614156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.614425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.615056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.616664 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.627626 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1192/2345] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:01.631890 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.642027 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55475e32-f5d5-4d7d-a247-5d94d6de497e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.646717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.647363 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.647642 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.647930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] PATH_INFO: `/v3/projects/7e7d9912fcd14c08929a7c10d7482ec6/users/b0660aa31ef94324aa6e575e6efc35e0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.663266 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7d9912fcd14c08929a7c10d7482ec6, user_id=b0660aa31ef94324aa6e575e6efc35e0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.665977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.679169 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1154/2346] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:01 2022] PUT /identity/v3/projects/7e7d9912fcd14c08929a7c10d7482ec6/users/b0660aa31ef94324aa6e575e6efc35e0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:01.683075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.693042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afae34-054b-4ba7-8c9b-c9ee176c47de admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.736933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:01.749501 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0965e060-f0ff-42c3-9abb-37acfbd2472d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.750104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e060-f0ff-42c3-9abb-37acfbd2472d admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.750388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e060-f0ff-42c3-9abb-37acfbd2472d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.750659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0965e060-f0ff-42c3-9abb-37acfbd2472d admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.751301 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965e060-f0ff-42c3-9abb-37acfbd2472d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.752962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0965e060-f0ff-42c3-9abb-37acfbd2472d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.762770 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1193/2347] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:01.766923 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.777118 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445115fb-7688-49f8-b34a-1feeea6f1129 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.782591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.783280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.783578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.783851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] PATH_INFO: `/v3/projects/7e7d9912fcd14c08929a7c10d7482ec6/users/b0660aa31ef94324aa6e575e6efc35e0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.786007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e7d9912fcd14c08929a7c10d7482ec6, user_id=b0660aa31ef94324aa6e575e6efc35e0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:01.788703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:01.799588 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1155/2348] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:01 2022] PUT /identity/v3/projects/7e7d9912fcd14c08929a7c10d7482ec6/users/b0660aa31ef94324aa6e575e6efc35e0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:01.809830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.810263 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.810659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.841719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] MFA Rules not processed for user `b0660aa31ef94324aa6e575e6efc35e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:01.842938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:01.912447 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1194/2349] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:01.924233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:01.925977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:01.964983 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:01.984665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642b89c5-ee0c-4cb2-8c19-7fa2083dc46b admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:01.988861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:01.989461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:01.989737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:01.990226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:01.999088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:02.000625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:02.010334 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1156/2350] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:02.052390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:02.053749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:02.065520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14469db2-bea0-42f9-8e44-ed14e1ab7ec8 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:02.069040 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:02.069629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:02.069907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:02.070171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:02.124648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:02.126116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:02.134943 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1195/2351] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:03.616979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:03.618959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:03.682585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8158d9a-555d-4170-9697-46d5f71ce849 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:03.686640 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b7083-045c-4386-a987-419ad2eff7e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:03.687254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7083-045c-4386-a987-419ad2eff7e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:03.687529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7083-045c-4386-a987-419ad2eff7e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:03.688361 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b7083-045c-4386-a987-419ad2eff7e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:03.690446 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-986b7083-045c-4386-a987-419ad2eff7e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:03.760410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b7083-045c-4386-a987-419ad2eff7e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:03.761858 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986b7083-045c-4386-a987-419ad2eff7e5 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:03.772104 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1157/2352] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:05.086537 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:05.098794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f372d1c6-9f4b-4ef1-909a-7fcdc83ee55a service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:05.104443 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:05.105095 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:05.105378 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:05.105651 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] PATH_INFO: `/v3/users/2720cc702fcc413dbb2edf75a6b9ef6d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:05.107142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2720cc702fcc413dbb2edf75a6b9ef6d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:05.109115 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:05.184207 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2720cc702fcc413dbb2edf75a6b9ef6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.194351 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2720cc702fcc413dbb2edf75a6b9ef6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.205708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2720cc702fcc413dbb2edf75a6b9ef6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.207961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:05.227821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2720cc702fcc413dbb2edf75a6b9ef6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.228731 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2720cc702fcc413dbb2edf75a6b9ef6d'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.236932 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1196/2353] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:05 2022] DELETE /identity/v3/users/2720cc702fcc413dbb2edf75a6b9ef6d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:05.241286 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:05.242163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:05.242610 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:05.276771 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:05.278318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:05.365507 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1158/2354] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:05.377536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:05.379971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:05.454400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c176d62d-b920-4a45-a0e5-307576434108 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:05.458473 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:05.459312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:05.459641 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:05.460013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:05.472224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:05.474011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:05.499590 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1197/2355] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:05.846149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:05.847608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:05.860516 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f65fce-6d69-4bb1-9fa8-f98b41045180 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:05.864863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:05.865499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:05.865774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:05.866045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] PATH_INFO: `/v3/projects/bb9bd828b0814dcf9a487330efa76638` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:05.867145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb9bd828b0814dcf9a487330efa76638)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:05.869080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:05.895492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:05.929106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb9bd828b0814dcf9a487330efa76638', 'request_id': 'req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.938354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb9bd828b0814dcf9a487330efa76638', 'request_id': 'req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.945966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb9bd828b0814dcf9a487330efa76638', 'request_id': 'req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.959002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] The token cache is being invalidate because project bb9bd828b0814dcf9a487330efa76638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:05.959590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.960066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:05.962142 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1159/2356] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:05 2022] DELETE /identity/v3/projects/bb9bd828b0814dcf9a487330efa76638 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:05.966035 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:05.977114 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f23502-94e4-427c-b181-a85941a7aa1b service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:06.043522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:06.044246 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:06.044632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:06.044970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] PATH_INFO: `/v3/users/4e16ec9271d34d1aa1e878fcd6a6ecc1` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:06.046250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e16ec9271d34d1aa1e878fcd6a6ecc1)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:06.048209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:06.103815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e16ec9271d34d1aa1e878fcd6a6ecc1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.115918 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e16ec9271d34d1aa1e878fcd6a6ecc1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.124949 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e16ec9271d34d1aa1e878fcd6a6ecc1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.142780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e16ec9271d34d1aa1e878fcd6a6ecc1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.143561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e16ec9271d34d1aa1e878fcd6a6ecc1'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.151272 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1198/2357] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:05 2022] DELETE /identity/v3/users/4e16ec9271d34d1aa1e878fcd6a6ecc1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:06.595087 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:06.597705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:06.619318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8eebf-b0ed-4b26-82c4-bcb0272e5d20 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:06.688627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:06.689285 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:06.689566 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:06.689841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] PATH_INFO: `/v3/projects/93e229d777994471b526b6b65ba1c82e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:06.691088 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93e229d777994471b526b6b65ba1c82e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:06.693529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:06.756210 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93e229d777994471b526b6b65ba1c82e', 'request_id': 'req-37d26384-f757-490a-8620-7b52aa737fe7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.770310 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93e229d777994471b526b6b65ba1c82e', 'request_id': 'req-37d26384-f757-490a-8620-7b52aa737fe7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.778042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93e229d777994471b526b6b65ba1c82e', 'request_id': 'req-37d26384-f757-490a-8620-7b52aa737fe7'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.792558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] The token cache is being invalidate because project 93e229d777994471b526b6b65ba1c82e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:06.792900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.793561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.796408 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1160/2358] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:06 2022] DELETE /identity/v3/projects/93e229d777994471b526b6b65ba1c82e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:06.801216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:06.802885 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:06.822223 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:06.836355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa801b9-7370-47f4-8fe9-1d1816c56d6f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:06.918636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:06.919279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:06.919567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:06.919825 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] PATH_INFO: `/v3/users/b0660aa31ef94324aa6e575e6efc35e0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:06.920971 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0660aa31ef94324aa6e575e6efc35e0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:06.924226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:06.974719 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0660aa31ef94324aa6e575e6efc35e0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.981907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0660aa31ef94324aa6e575e6efc35e0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:06.990033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0660aa31ef94324aa6e575e6efc35e0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.004417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0660aa31ef94324aa6e575e6efc35e0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.005155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0660aa31ef94324aa6e575e6efc35e0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.011226 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1199/2359] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:06 2022] DELETE /identity/v3/users/b0660aa31ef94324aa6e575e6efc35e0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:07.465780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:07.467396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:07.483123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d26384-f757-490a-8620-7b52aa737fe7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:07.537436 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:07.551986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:07.554612 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:07.568632 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:07.569262 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:07.569537 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:07.569801 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] PATH_INFO: `/v3/projects/7e7d9912fcd14c08929a7c10d7482ec6` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:07.570912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e7d9912fcd14c08929a7c10d7482ec6)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:07.572822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:07.624479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e7d9912fcd14c08929a7c10d7482ec6', 'request_id': 'req-e0eab1f1-16a4-46da-9194-3f74b742f423'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.628902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683a510-dccc-40eb-940f-b72f33d489e6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:07.632770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e7d9912fcd14c08929a7c10d7482ec6', 'request_id': 'req-e0eab1f1-16a4-46da-9194-3f74b742f423'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.640418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e7d9912fcd14c08929a7c10d7482ec6', 'request_id': 'req-e0eab1f1-16a4-46da-9194-3f74b742f423'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.652488 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] The token cache is being invalidate because project 7e7d9912fcd14c08929a7c10d7482ec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:07.652776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.653418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e0eab1f1-16a4-46da-9194-3f74b742f423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.654894 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1161/2360] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:07 2022] DELETE /identity/v3/projects/7e7d9912fcd14c08929a7c10d7482ec6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:07.660603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:07.660766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638246eb-d97c-4298-96b7-a06b87d70ca8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:07.660944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638246eb-d97c-4298-96b7-a06b87d70ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:07.661217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638246eb-d97c-4298-96b7-a06b87d70ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:07.680896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-638246eb-d97c-4298-96b7-a06b87d70ca8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:07.682095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-638246eb-d97c-4298-96b7-a06b87d70ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:07.694091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:07.694726 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:07.695004 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:07.695279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] PATH_INFO: `/v3/users/11b0f829f5654010b2bd1a76888d7ebf` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:07.696459 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11b0f829f5654010b2bd1a76888d7ebf)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:07.705538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:07.743380 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1162/2361] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:07.754097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baea717-e80d-4412-8b7e-f1bd385f8246 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:07.754380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baea717-e80d-4412-8b7e-f1bd385f8246 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:07.754650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baea717-e80d-4412-8b7e-f1bd385f8246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:07.755554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11b0f829f5654010b2bd1a76888d7ebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.762670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11b0f829f5654010b2bd1a76888d7ebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.770544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11b0f829f5654010b2bd1a76888d7ebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.776499 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2baea717-e80d-4412-8b7e-f1bd385f8246 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:07.777707 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2baea717-e80d-4412-8b7e-f1bd385f8246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:07.782957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11b0f829f5654010b2bd1a76888d7ebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.783735 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11b0f829f5654010b2bd1a76888d7ebf'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:07.786023 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1163/2362] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:07.789823 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1200/2363] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:07 2022] DELETE /identity/v3/users/11b0f829f5654010b2bd1a76888d7ebf => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:07.790072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baea717-e80d-4412-8b7e-f1bd385f8246 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:07.799042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baea717-e80d-4412-8b7e-f1bd385f8246 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:07.871174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:07.871834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:07.872113 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:07.872413 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:07.873054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:07.873404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:07.875573 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:07.884493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:07.888479 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1164/2364] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:05:07.893143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:07.894755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:07.908232 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9336ac-532b-4bd0-ac3e-adb3702a9955 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:07.913071 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:07.913728 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:07.914006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:07.914320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:07.914985 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:07.916977 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:07.948129 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1201/2365] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:07 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:07.950193 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:07.962418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44971366-3150-4e85-a46e-47f30de711b7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:07.977458 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:07.979363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:08.033371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.034064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.034464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.035221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.035514 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.037733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.054538 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15887c2a-249d-4f4d-a230-50566b3ebc50 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.059903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.060817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.061169 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.061529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] PATH_INFO: `/v3/projects/7103bcb49d5c4e8b958fba3b1309a881` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.063466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7103bcb49d5c4e8b958fba3b1309a881)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.065520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.071925 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1165/2366] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:07 2022] POST /identity/v3/users => generated 291 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:08.076502 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.089116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe27059-a5b1-4e2b-a656-5764379ba04d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.096045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.096702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.097049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.097317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.098042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.100231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.104594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:08.117002 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1166/2367] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:08.120827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.130009 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7103bcb49d5c4e8b958fba3b1309a881', 'request_id': 'req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:08.136766 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e8285-68b9-4501-b2fd-ff05b03c2e47 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.141932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7103bcb49d5c4e8b958fba3b1309a881', 'request_id': 'req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:08.153324 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7103bcb49d5c4e8b958fba3b1309a881', 'request_id': 'req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:08.172020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] The token cache is being invalidate because project 7103bcb49d5c4e8b958fba3b1309a881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:08.172331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:08.173043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-47d2c3ed-68de-4d67-bd8e-ae3cdffaf3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:08.175255 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1202/2368] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:07 2022] DELETE /identity/v3/projects/7103bcb49d5c4e8b958fba3b1309a881 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:08.180450 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e3a3c-aea4-4509-8959-ae70cfbe6ab7 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.180729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e3a3c-aea4-4509-8959-ae70cfbe6ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.181050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e3a3c-aea4-4509-8959-ae70cfbe6ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.205587 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6c1e3a3c-aea4-4509-8959-ae70cfbe6ab7 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:08.207356 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6c1e3a3c-aea4-4509-8959-ae70cfbe6ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:08.237409 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.238213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.238557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.238904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.239799 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.241484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.253946 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1167/2369] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:08.257822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.259613 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:08.273618 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f801c33-2237-41e3-8e71-54caa7a5e5bb admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.287097 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001e9e68-65f1-4c90-8d5c-36d8c1551477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.288076 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e9e68-65f1-4c90-8d5c-36d8c1551477 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.288476 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e9e68-65f1-4c90-8d5c-36d8c1551477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.288826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001e9e68-65f1-4c90-8d5c-36d8c1551477 admin admin] PATH_INFO: `/v3/projects/71f9cd06cdff4fd0be15427c2affa8cc/users/3d443482cffb449694f3f9c02e6e8826/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.290317 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1203/2370] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:08.298810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727bb29-0a2d-4e14-be06-7bff137a9615 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.299195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727bb29-0a2d-4e14-be06-7bff137a9615 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.299571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727bb29-0a2d-4e14-be06-7bff137a9615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.308896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e9e68-65f1-4c90-8d5c-36d8c1551477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f9cd06cdff4fd0be15427c2affa8cc, user_id=3d443482cffb449694f3f9c02e6e8826, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.313407 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001e9e68-65f1-4c90-8d5c-36d8c1551477 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.321033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-0727bb29-0a2d-4e14-be06-7bff137a9615 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:08.322469 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-0727bb29-0a2d-4e14-be06-7bff137a9615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:08.330515 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1204/2371] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:08.334887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727bb29-0a2d-4e14-be06-7bff137a9615 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.337190 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1168/2372] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:08 2022] PUT /identity/v3/projects/71f9cd06cdff4fd0be15427c2affa8cc/users/3d443482cffb449694f3f9c02e6e8826/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:08.348038 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727bb29-0a2d-4e14-be06-7bff137a9615 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.348768 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.349054 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.349326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.377260 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] MFA Rules not processed for user `3d443482cffb449694f3f9c02e6e8826`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:08.378627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:08.404844 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.405456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.405738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.406014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.407293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:08.407495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.409198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.418170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:08.420994 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1205/2373] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:05:08.425893 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.436044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654ebd-4398-44e1-a565-976fcfd28a77 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.440763 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.441338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.441618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.441886 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.442499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.444495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.446497 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1169/2374] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:08.457470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.458834 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:08.471890 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1206/2375] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:08.475808 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.501803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:08.520959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55321290-8429-4b72-8204-38b712a5ef7a None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.541424 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2487a27e-03d2-439a-952c-a8332728b2b6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.546143 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.546747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.547590 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.547881 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.548570 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.550498 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.572116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf2da38-e18d-4468-9ae9-3bf49e9a1e36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.572772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2da38-e18d-4468-9ae9-3bf49e9a1e36 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.573014 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1207/2376] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:08.573505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2da38-e18d-4468-9ae9-3bf49e9a1e36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.573505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf2da38-e18d-4468-9ae9-3bf49e9a1e36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.577235 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.587434 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a40bb6-76df-4ddb-b609-af2c2b153b90 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.592279 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.592789 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.593062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.593331 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.593997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.595602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.606202 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1208/2377] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:08.610125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.619461 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15273f0-6ba1-41f2-ba52-4b346dec6301 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.624021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.624631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.624907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.625734 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.626362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.627975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.637464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2da38-e18d-4468-9ae9-3bf49e9a1e36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.638796 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1209/2378] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:08.638907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf2da38-e18d-4468-9ae9-3bf49e9a1e36 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.644804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.651496 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1170/2379] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:05:08.654685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b9a393-20a2-4e7b-8719-8ae1ad74c61e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.659479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.660125 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.660435 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.660799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] PATH_INFO: `/v3/projects/d603a305270945de898af14beb1734a0/users/452665f2613f4b9b98fc2d2d924c8f1e/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.675778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d603a305270945de898af14beb1734a0, user_id=452665f2613f4b9b98fc2d2d924c8f1e, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.678544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.691829 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1210/2380] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:08 2022] PUT /identity/v3/projects/d603a305270945de898af14beb1734a0/users/452665f2613f4b9b98fc2d2d924c8f1e/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:08.704102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef8aef-9b71-41a6-9bb4-34ef7ebb731b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.704422 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef8aef-9b71-41a6-9bb4-34ef7ebb731b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.704699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef8aef-9b71-41a6-9bb4-34ef7ebb731b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.731742 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-71ef8aef-9b71-41a6-9bb4-34ef7ebb731b None None] MFA Rules not processed for user `452665f2613f4b9b98fc2d2d924c8f1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:08.733102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-71ef8aef-9b71-41a6-9bb4-34ef7ebb731b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:08.774377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:08.799364 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1171/2381] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:08.803722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.878274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342742b5-2798-4990-a30a-a4d4bc98d86b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:08.883417 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:08.884020 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:08.884321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:08.884595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:08.885226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:08.887113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:08.921737 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1211/2382] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/projects => generated 373 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:08.924257 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef8aef-9b71-41a6-9bb4-34ef7ebb731b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:08.936789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef8aef-9b71-41a6-9bb4-34ef7ebb731b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:09.001130 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:09.001750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.002032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.002393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.002929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:09.005300 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:09.028729 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1172/2383] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:08 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:09.030131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:09.040992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78424ad1-d2aa-48f6-9a19-a5a53122b972 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:09.045729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:09.046323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.046604 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.046906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.047548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:09.049287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:09.060047 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1212/2384] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:09.064213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:09.074001 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a754db-3af1-41c3-bb3e-240530451ab6 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:09.079015 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:09.079651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.079928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.080200 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] PATH_INFO: `/v3/projects/8f6e3f870a864895822901aa5a9a7b35/users/bda1b4bba2694b4e8f8ef8bc0ceaf3a0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.095520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6e3f870a864895822901aa5a9a7b35, user_id=bda1b4bba2694b4e8f8ef8bc0ceaf3a0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:09.098231 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:09.111071 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1173/2385] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:09 2022] PUT /identity/v3/projects/8f6e3f870a864895822901aa5a9a7b35/users/bda1b4bba2694b4e8f8ef8bc0ceaf3a0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:09.114966 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:09.124767 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d3c77-2981-472c-b735-62837e4b3c17 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:09.162820 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:09.181958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030db084-2ac7-4948-a415-9d194ee9e7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:09.182558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030db084-2ac7-4948-a415-9d194ee9e7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.182834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030db084-2ac7-4948-a415-9d194ee9e7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.183104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030db084-2ac7-4948-a415-9d194ee9e7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.183730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030db084-2ac7-4948-a415-9d194ee9e7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:09.185349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030db084-2ac7-4948-a415-9d194ee9e7e1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:09.195496 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1213/2386] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:09.199602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:09.210341 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d7f1c-614b-44b4-86b8-faf1455cecdc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:09.214872 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:09.215509 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.215783 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.216047 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] PATH_INFO: `/v3/projects/8f6e3f870a864895822901aa5a9a7b35/users/bda1b4bba2694b4e8f8ef8bc0ceaf3a0/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.218041 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f6e3f870a864895822901aa5a9a7b35, user_id=bda1b4bba2694b4e8f8ef8bc0ceaf3a0, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:09.220716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:09.230607 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1174/2387] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:09 2022] PUT /identity/v3/projects/8f6e3f870a864895822901aa5a9a7b35/users/bda1b4bba2694b4e8f8ef8bc0ceaf3a0/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:09.240813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bb9fb-4caa-4daa-b85e-023270d8e017 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.241100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bb9fb-4caa-4daa-b85e-023270d8e017 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.241376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2bb9fb-4caa-4daa-b85e-023270d8e017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.269704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bf2bb9fb-4caa-4daa-b85e-023270d8e017 None None] MFA Rules not processed for user `bda1b4bba2694b4e8f8ef8bc0ceaf3a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:09.270903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bf2bb9fb-4caa-4daa-b85e-023270d8e017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:09.336996 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1214/2388] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:09.350051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:09.411072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e508bd-6044-4baa-b966-8b705af82134 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:09.414871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e45491-c0b9-4ef1-8a8c-78025d145224 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:09.415486 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e45491-c0b9-4ef1-8a8c-78025d145224 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.415763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e45491-c0b9-4ef1-8a8c-78025d145224 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.416029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e45491-c0b9-4ef1-8a8c-78025d145224 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.424950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e45491-c0b9-4ef1-8a8c-78025d145224 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:09.426369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e45491-c0b9-4ef1-8a8c-78025d145224 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:09.435817 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1175/2389] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:09.516589 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bb9fb-4caa-4daa-b85e-023270d8e017 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:09.526010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2bb9fb-4caa-4daa-b85e-023270d8e017 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:09.529682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:09.530308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:09.530611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:09.530856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:09.588082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:09.589812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:09.601042 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1215/2390] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:13.304221 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d4027-180a-46dd-8cf5-c3f12c535fa0 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:13.304775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d4027-180a-46dd-8cf5-c3f12c535fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:13.304775 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d4027-180a-46dd-8cf5-c3f12c535fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:13.323929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-381d4027-180a-46dd-8cf5-c3f12c535fa0 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:13.325143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-381d4027-180a-46dd-8cf5-c3f12c535fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:13.354521 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:13.381605 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1176/2391] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 01:05:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:13.425670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:13.427110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:13.492192 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:13.498591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60bfbd-565a-408b-becb-eedead5eb7cc service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:13.504729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:13.505437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:13.505835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:13.506176 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:13.514998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:13.516591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:13.524843 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1216/2392] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:05:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:05:14.040177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c143d9b-f3ec-4c74-b143-c7695736b56f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.040484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c143d9b-f3ec-4c74-b143-c7695736b56f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.040754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c143d9b-f3ec-4c74-b143-c7695736b56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.062190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:14.063526 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4c143d9b-f3ec-4c74-b143-c7695736b56f None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:14.065675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4c143d9b-f3ec-4c74-b143-c7695736b56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.073408 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1177/2393] 10.222.0.39 () {58 vars in 967 bytes} [Sun Oct 23 01:05:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:14.086976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c143d9b-f3ec-4c74-b143-c7695736b56f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:14.101030 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c143d9b-f3ec-4c74-b143-c7695736b56f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:14.108633 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafec67c-88f1-4cf6-8549-633c62b8f191 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:14.109656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafec67c-88f1-4cf6-8549-633c62b8f191 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.109947 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafec67c-88f1-4cf6-8549-633c62b8f191 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.110233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafec67c-88f1-4cf6-8549-633c62b8f191 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.119615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafec67c-88f1-4cf6-8549-633c62b8f191 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:14.121276 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafec67c-88f1-4cf6-8549-633c62b8f191 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:14.125804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d326718-89f8-4262-b270-c871e0214e4c service swift] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:14.129542 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1178/2394] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 01:05:14.130670 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:14.131322 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.131582 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.131908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] PATH_INFO: `/v3/users/3d443482cffb449694f3f9c02e6e8826` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.133025 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d443482cffb449694f3f9c02e6e8826)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:14.134779 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:14.186403 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d443482cffb449694f3f9c02e6e8826'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.194234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d443482cffb449694f3f9c02e6e8826'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.201355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d443482cffb449694f3f9c02e6e8826'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.214215 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d443482cffb449694f3f9c02e6e8826'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.215059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d443482cffb449694f3f9c02e6e8826'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.222108 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1217/2395] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:14 2022] DELETE /identity/v3/users/3d443482cffb449694f3f9c02e6e8826 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:14.226441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52ad9-7fec-43c7-81a7-42256d9c0110 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.226724 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52ad9-7fec-43c7-81a7-42256d9c0110 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.227013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a52ad9-7fec-43c7-81a7-42256d9c0110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.247060 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e1a52ad9-7fec-43c7-81a7-42256d9c0110 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:14.248247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e1a52ad9-7fec-43c7-81a7-42256d9c0110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.307151 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1179/2396] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:14.317190 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:14.318579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.366861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.367247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.367589 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.377795 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2e9335-4df8-405a-ac2c-5283d6334417 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:14.381697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:14.382299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.382574 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.382847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.387315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:14.388529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.392017 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:14.394132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:14.403492 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1218/2397] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:14.444547 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1180/2398] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 01:05:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:14.457493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:14.459023 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.473622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:14.531953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878a7508-1cf4-4184-a9ed-673313a32765 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:14.537264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b19514-8c8b-4e03-9702-443a1eae7fed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:14.537869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b19514-8c8b-4e03-9702-443a1eae7fed service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.538142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b19514-8c8b-4e03-9702-443a1eae7fed service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.538408 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b19514-8c8b-4e03-9702-443a1eae7fed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.547293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b19514-8c8b-4e03-9702-443a1eae7fed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:14.548911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b19514-8c8b-4e03-9702-443a1eae7fed service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:14.557580 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1219/2399] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:05:14.694535 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:14.695925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.710945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9eab1a-ceaf-46f8-a5b7-1149fbd4d13b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:14.716281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:14.717213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.717540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.717749 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] PATH_INFO: `/v3/projects/71f9cd06cdff4fd0be15427c2affa8cc` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.719034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f9cd06cdff4fd0be15427c2affa8cc)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:14.721220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:14.771777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f9cd06cdff4fd0be15427c2affa8cc', 'request_id': 'req-d93f9483-38d3-46c7-bc01-3b3e627df461'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.779868 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f9cd06cdff4fd0be15427c2affa8cc', 'request_id': 'req-d93f9483-38d3-46c7-bc01-3b3e627df461'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.783912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:14.786385 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f9cd06cdff4fd0be15427c2affa8cc', 'request_id': 'req-d93f9483-38d3-46c7-bc01-3b3e627df461'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.800561 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] The token cache is being invalidate because project 71f9cd06cdff4fd0be15427c2affa8cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:14.800892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.801671 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d93f9483-38d3-46c7-bc01-3b3e627df461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:14.803366 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1181/2400] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:14 2022] DELETE /identity/v3/projects/71f9cd06cdff4fd0be15427c2affa8cc => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:14.808164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3b99f-f77d-4396-835b-1d1b209ad44d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.808794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3b99f-f77d-4396-835b-1d1b209ad44d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.809069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3b99f-f77d-4396-835b-1d1b209ad44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.828887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4cf3b99f-f77d-4396-835b-1d1b209ad44d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:14.830242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4cf3b99f-f77d-4396-835b-1d1b209ad44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.894716 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1220/2401] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:14.902431 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8052098-a1f3-47c7-aff5-cb4bbc1ce045 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.902687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8052098-a1f3-47c7-aff5-cb4bbc1ce045 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.902930 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8052098-a1f3-47c7-aff5-cb4bbc1ce045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.910003 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.910287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.910600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.922886 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e8052098-a1f3-47c7-aff5-cb4bbc1ce045 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:14.924061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e8052098-a1f3-47c7-aff5-cb4bbc1ce045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.930175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:14.931339 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:14.932289 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1182/2402] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:14.936289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8052098-a1f3-47c7-aff5-cb4bbc1ce045 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:14.946152 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8052098-a1f3-47c7-aff5-cb4bbc1ce045 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:14.950638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:14.951217 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.951494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.951759 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.952415 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:14.952715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:14.954279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:14.965455 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1183/2403] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:05:14.969507 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:14.978682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09db4193-ec33-478c-bb92-29921175e0ee admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:14.983364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:14.983938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:14.984209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:14.984501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:14.985112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:14.987549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:14.988107 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1221/2404] 10.222.0.39 () {58 vars in 967 bytes} [Sun Oct 23 01:05:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:15.000158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.001507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:15.017194 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1184/2405] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:14 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:15.021447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.052998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:15.075210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa85d4a-f46e-4966-8ef2-72e80b5e4a4d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.082456 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2606543c-1c12-46f4-af8b-1eb3c9e2db37 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.087211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.087808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.088084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.088388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.089008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.090955 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.115438 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1185/2406] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:15 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:15.116518 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.126254 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7d0bfc-3ca4-4c55-8859-2929b3e24609 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.131433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.132029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.132353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.132615 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.133357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.134938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.136809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b1f0dc-28a1-4be2-a4a9-3bcc08131715 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.137953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1f0dc-28a1-4be2-a4a9-3bcc08131715 service swift] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.138228 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1f0dc-28a1-4be2-a4a9-3bcc08131715 service swift] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.138497 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b1f0dc-28a1-4be2-a4a9-3bcc08131715 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.148808 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1186/2407] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:15.153340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.167716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af46e53-63b0-412a-96e1-b5b9f52dd1d5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.172793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.173469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.173669 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.173938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.175163 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.176779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.188903 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1187/2408] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:15.195655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.207368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1f0dc-28a1-4be2-a4a9-3bcc08131715 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.209033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b1f0dc-28a1-4be2-a4a9-3bcc08131715 service swift] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.213386 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f13f6d-8411-4a06-a2e0-a9a2b38f19ea admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.218296 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1222/2409] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 01:05:15.219412 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.220026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.220371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.220643 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] PATH_INFO: `/v3/projects/7a718da62ffa4afcacbe2214271d9bc4/users/86a6c9a854bb44b6a64bc95ca8332b1c/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.237593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a718da62ffa4afcacbe2214271d9bc4, user_id=86a6c9a854bb44b6a64bc95ca8332b1c, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.240554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.257013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1188/2410] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:15 2022] PUT /identity/v3/projects/7a718da62ffa4afcacbe2214271d9bc4/users/86a6c9a854bb44b6a64bc95ca8332b1c/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:15.267751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bdad-bf8c-4ae7-ac95-cc0821b6d684 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.268183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bdad-bf8c-4ae7-ac95-cc0821b6d684 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.268505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc6bdad-bf8c-4ae7-ac95-cc0821b6d684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.298043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9dc6bdad-bf8c-4ae7-ac95-cc0821b6d684 None None] MFA Rules not processed for user `86a6c9a854bb44b6a64bc95ca8332b1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:15.300021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9dc6bdad-bf8c-4ae7-ac95-cc0821b6d684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:15.416520 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1223/2411] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:15.420819 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.490984 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:15.506174 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6d3a0e-b398-4917-8eb8-7de5df8a6133 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.514393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.515075 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.515357 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.515625 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.516275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.518831 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.553354 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1189/2412] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:15 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:15.558899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6bdad-bf8c-4ae7-ac95-cc0821b6d684 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.571247 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc6bdad-bf8c-4ae7-ac95-cc0821b6d684 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.603203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:15.646776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.647392 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.647684 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.648013 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.648688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.650936 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.674641 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1224/2413] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:15 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:15.678715 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.689175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c5395-01be-4017-9c57-66268953dc3d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.696222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.696876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.697157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.697465 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.698108 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.699689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.714409 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1190/2414] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:15.716057 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.730830 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909f6638-22c1-46f3-a8aa-8ace908baabc admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.737144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fb9d8b-7ed6-4428-97b6-2b91db82c5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.737306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9d8b-7ed6-4428-97b6-2b91db82c5fa admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.737662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9d8b-7ed6-4428-97b6-2b91db82c5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.737942 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fb9d8b-7ed6-4428-97b6-2b91db82c5fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.738610 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb9d8b-7ed6-4428-97b6-2b91db82c5fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.740312 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fb9d8b-7ed6-4428-97b6-2b91db82c5fa admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.751513 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1225/2415] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:15.756757 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.768653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ac740c-5fbd-47e0-8ec9-206403e3f743 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:15.773490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:15.774175 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.774455 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.774728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] PATH_INFO: `/v3/projects/52df91f07d364b34a197b9b64cedc20a/users/bfd2963c66d04ae8a53db92b87191cae/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.791039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52df91f07d364b34a197b9b64cedc20a, user_id=bfd2963c66d04ae8a53db92b87191cae, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:15.793734 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:15.812333 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1191/2416] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:15 2022] PUT /identity/v3/projects/52df91f07d364b34a197b9b64cedc20a/users/bfd2963c66d04ae8a53db92b87191cae/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:15.822449 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd217fc-95ea-4a60-8186-98e049b28920 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:15.822835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd217fc-95ea-4a60-8186-98e049b28920 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:15.823180 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd217fc-95ea-4a60-8186-98e049b28920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:15.854532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-bcd217fc-95ea-4a60-8186-98e049b28920 None None] MFA Rules not processed for user `bfd2963c66d04ae8a53db92b87191cae`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:15.855851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bcd217fc-95ea-4a60-8186-98e049b28920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:15.927269 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1226/2417] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:15.931732 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:15.995809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ab41a2-898e-4bf2-b49f-9ea14f1ba940 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:16.000547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:16.001145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.001418 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.001686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.002833 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:16.005102 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:16.037856 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1192/2418] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:15 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:16.041625 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd217fc-95ea-4a60-8186-98e049b28920 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:16.054809 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd217fc-95ea-4a60-8186-98e049b28920 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:16.119722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:16.120598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.120774 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.121027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.121658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:16.123680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:16.152516 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1227/2419] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:16 2022] POST /identity/v3/users => generated 299 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:16.155057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:16.166145 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cca0fc-20de-40ee-8e13-84ab7bb039bf admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:16.170904 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:16.171714 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.171995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.172273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.172966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:16.175106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:16.190876 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1193/2420] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:16.192259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:16.203521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4c4ed-6445-49dd-80a7-f0014064105b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:16.208585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:16.209260 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.209533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.209801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] PATH_INFO: `/v3/projects/67bd7b7119744a82a4cc590503e5affa/users/65f206a55ef74646aff5158af6a823ec/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.226399 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bd7b7119744a82a4cc590503e5affa, user_id=65f206a55ef74646aff5158af6a823ec, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:16.229164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:16.231961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:16.245410 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1228/2421] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:16 2022] PUT /identity/v3/projects/67bd7b7119744a82a4cc590503e5affa/users/65f206a55ef74646aff5158af6a823ec/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:16.249645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:16.261536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7fa95-dd96-463c-90c7-06ac66bc0527 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:16.322763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca2d965-8008-4803-ba45-d26fbd216053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:16.323377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2d965-8008-4803-ba45-d26fbd216053 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.323656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2d965-8008-4803-ba45-d26fbd216053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.323927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca2d965-8008-4803-ba45-d26fbd216053 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.324594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2d965-8008-4803-ba45-d26fbd216053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:16.326279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca2d965-8008-4803-ba45-d26fbd216053 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:16.341618 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1194/2422] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:16.342648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:16.352986 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8897ed90-a9a1-4040-bc87-179286bc89ae admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:16.357601 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:16.358245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.358526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.358799 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] PATH_INFO: `/v3/projects/67bd7b7119744a82a4cc590503e5affa/users/65f206a55ef74646aff5158af6a823ec/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.360873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] RBAC: Authorizing `identity:create_grant(project_id=67bd7b7119744a82a4cc590503e5affa, user_id=65f206a55ef74646aff5158af6a823ec, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:16.364027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:16.374438 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1229/2423] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:16 2022] PUT /identity/v3/projects/67bd7b7119744a82a4cc590503e5affa/users/65f206a55ef74646aff5158af6a823ec/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:16.385815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.386133 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.386430 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.408945 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:16.417847 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] MFA Rules not processed for user `65f206a55ef74646aff5158af6a823ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:16.419058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:16.492710 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1195/2424] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:16.506579 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:16.508314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:16.565683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbfdaeb-5b3b-4246-9494-0cf7126c5acd admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:16.569363 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e534d3-57e0-4a43-8c22-a397708a4964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:16.569952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e534d3-57e0-4a43-8c22-a397708a4964 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:16.570227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e534d3-57e0-4a43-8c22-a397708a4964 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:16.570493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e534d3-57e0-4a43-8c22-a397708a4964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:16.635688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e534d3-57e0-4a43-8c22-a397708a4964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:16.637157 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e534d3-57e0-4a43-8c22-a397708a4964 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:16.646321 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1230/2425] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:17.523685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:17.525111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:17.587068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2224533-0585-42f9-9b5b-ab9860ee43d2 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:17.591664 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:17.592072 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:17.592364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:17.592634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] PATH_INFO: `/v3/users/4520c2c66d38454a890f81cb82545f56` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:17.593747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4520c2c66d38454a890f81cb82545f56)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:17.595651 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:17.639575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4520c2c66d38454a890f81cb82545f56'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:17.640209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4520c2c66d38454a890f81cb82545f56'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:17.647569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4520c2c66d38454a890f81cb82545f56'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:17.659421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4520c2c66d38454a890f81cb82545f56'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:17.660058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4520c2c66d38454a890f81cb82545f56'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:17.672740 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1196/2426] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:17 2022] DELETE /identity/v3/users/4520c2c66d38454a890f81cb82545f56 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:17.677810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:17.678334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:17.678334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:17.697268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:17.698438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:17.754931 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1231/2427] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:17.764512 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:17.765899 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:17.784173 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:17.824976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d227118-4937-4f02-bb96-4da8365eb05c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:17.828563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e852707-04ea-411a-a043-a41f2e4939f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:17.829168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e852707-04ea-411a-a043-a41f2e4939f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:17.829445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e852707-04ea-411a-a043-a41f2e4939f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:17.829713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e852707-04ea-411a-a043-a41f2e4939f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:17.838761 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e852707-04ea-411a-a043-a41f2e4939f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:17.840166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e852707-04ea-411a-a043-a41f2e4939f1 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:17.849624 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1197/2428] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:18.037533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.039230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:18.055535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16286ca-dc2e-4cc0-9be2-95c161a5795f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.060894 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.062073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.062618 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.063104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] PATH_INFO: `/v3/projects/b44d7361c05546b0bc525a9dfe170f23` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.064532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44d7361c05546b0bc525a9dfe170f23)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.067303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.108847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:INSERT=2,keystone:SELECT=96,keystone:DELETE=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:18.133257 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44d7361c05546b0bc525a9dfe170f23', 'request_id': 'req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:18.142594 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44d7361c05546b0bc525a9dfe170f23', 'request_id': 'req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:18.154862 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44d7361c05546b0bc525a9dfe170f23', 'request_id': 'req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:18.169532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] The token cache is being invalidate because project b44d7361c05546b0bc525a9dfe170f23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:18.169926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:18.170667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:18.176770 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1232/2429] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:18 2022] DELETE /identity/v3/projects/b44d7361c05546b0bc525a9dfe170f23 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:18.178225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9654db8-32a7-4abb-bbc4-d116d1d56d1d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.178510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9654db8-32a7-4abb-bbc4-d116d1d56d1d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.178778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9654db8-32a7-4abb-bbc4-d116d1d56d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.199778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f9654db8-32a7-4abb-bbc4-d116d1d56d1d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:18.201252 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f9654db8-32a7-4abb-bbc4-d116d1d56d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:18.209062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.210637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:18.265491 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1198/2430] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:18.273234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d3f31-cab7-46f0-a8f3-628c7f1d4ada None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.273536 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d3f31-cab7-46f0-a8f3-628c7f1d4ada None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.273811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d3f31-cab7-46f0-a8f3-628c7f1d4ada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.290863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7a0a9-3c02-401d-b42a-e9c1f3af98c4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.294593 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e48d3f31-cab7-46f0-a8f3-628c7f1d4ada None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:18.294760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.295321 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.295432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e48d3f31-cab7-46f0-a8f3-628c7f1d4ada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:18.295597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.295927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.297687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:18.303931 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1199/2431] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:18.308275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d3f31-cab7-46f0-a8f3-628c7f1d4ada None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.318348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d3f31-cab7-46f0-a8f3-628c7f1d4ada None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.323657 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.324251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.324552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.324822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.325596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:18.325953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.327921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.344260 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1200/2432] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:05:18.345468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.356596 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73701a1-cddd-499b-a5d1-f9ecb888cb18 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.361485 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.362721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.362999 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.363265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.363897 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.365871 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.366135 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.367655 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.378691 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1233/2433] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:18 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:18.407501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1201/2434] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:18 2022] POST /identity/v3/projects => generated 391 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:18.409561 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.420193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed55471-f3b9-4c69-8189-50c36460169a service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.481433 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.482182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.482488 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.482756 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.483397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.485552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.506929 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1234/2435] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:18 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:18.511195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.521979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ab873-b622-4c3e-9f18-96e0c9b7b009 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.526447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.527042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.527299 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.527564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.528194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.530245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.541846 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1202/2436] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:18.545822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.551828 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:18.558484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a45a5d-0cf8-4202-ac07-3d7ec3d31503 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.565266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d3d86-d6f1-4107-ba59-f80b9e9bcaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.565867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d3d86-d6f1-4107-ba59-f80b9e9bcaad admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.566766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d3d86-d6f1-4107-ba59-f80b9e9bcaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.567044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d3d86-d6f1-4107-ba59-f80b9e9bcaad admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.567718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d3d86-d6f1-4107-ba59-f80b9e9bcaad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.569338 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d3d86-d6f1-4107-ba59-f80b9e9bcaad admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.584267 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1235/2437] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:18.585820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.600114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd6042-a8f7-4f3a-a2b9-d680f10821c7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.605665 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.606334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.606620 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.606892 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] PATH_INFO: `/v3/projects/92b75567eb6a4e98a4ee490dce493166/users/581a40a513e243d7a5fbd44f590fef95/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.622698 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b75567eb6a4e98a4ee490dce493166, user_id=581a40a513e243d7a5fbd44f590fef95, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.625460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.639803 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1203/2438] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:18 2022] PUT /identity/v3/projects/92b75567eb6a4e98a4ee490dce493166/users/581a40a513e243d7a5fbd44f590fef95/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:18.708813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.709411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.709411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.740198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] MFA Rules not processed for user `581a40a513e243d7a5fbd44f590fef95`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:18.741577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:18.812333 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1236/2439] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:18.822859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.861339 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:18.883952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3f053-2b5e-4da8-a3dc-e2fa10364686 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.889523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.890369 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.891794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.891794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.902031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:18.903914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:18.913511 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1204/2440] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:18.946189 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:18.947575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:18.959639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58ae981-3409-4a10-8124-46b5910ce5a0 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:18.963314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a244019-6f73-4394-ba1e-bdbd2cb660fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:18.963938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244019-6f73-4394-ba1e-bdbd2cb660fb service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:18.964216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244019-6f73-4394-ba1e-bdbd2cb660fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:18.964516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a244019-6f73-4394-ba1e-bdbd2cb660fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:18.966395 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2a244019-6f73-4394-ba1e-bdbd2cb660fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:19.029110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244019-6f73-4394-ba1e-bdbd2cb660fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:19.030540 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a244019-6f73-4394-ba1e-bdbd2cb660fb service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:19.041745 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1237/2441] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:18 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:23.063907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:23.065380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:23.130366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadc0833-9808-4809-aef9-8257a53e34c7 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:23.134940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:23.135567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:23.135845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:23.136116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] PATH_INFO: `/v3/users/86a6c9a854bb44b6a64bc95ca8332b1c` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:23.137295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a6c9a854bb44b6a64bc95ca8332b1c)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:23.139314 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:23.187125 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a6c9a854bb44b6a64bc95ca8332b1c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.187791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a6c9a854bb44b6a64bc95ca8332b1c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.195123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a6c9a854bb44b6a64bc95ca8332b1c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.207318 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a6c9a854bb44b6a64bc95ca8332b1c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.207959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a6c9a854bb44b6a64bc95ca8332b1c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.219782 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1205/2442] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:23 2022] DELETE /identity/v3/users/86a6c9a854bb44b6a64bc95ca8332b1c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:23.224901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:23.225357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:23.225509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:23.243961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:23.245224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:23.302610 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1238/2443] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:23.312084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:23.313493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:23.376623 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30f86d-efba-4096-bf81-86830973d0b9 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:23.380863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:23.381470 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:23.381750 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:23.382022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:23.391303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:23.392764 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:23.397439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:23.405312 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1206/2444] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:05:23.678566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:23.680201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:23.692743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465b66d-88e9-4adf-9329-66747b0d941c None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:23.697383 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:23.698014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:23.698295 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:23.698566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] PATH_INFO: `/v3/projects/7a718da62ffa4afcacbe2214271d9bc4` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:23.699703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a718da62ffa4afcacbe2214271d9bc4)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:23.701629 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:23.712554 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=100 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:23.748721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a718da62ffa4afcacbe2214271d9bc4', 'request_id': 'req-7f82c9c9-fb41-413b-830d-2f2024cfa13f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.756323 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a718da62ffa4afcacbe2214271d9bc4', 'request_id': 'req-7f82c9c9-fb41-413b-830d-2f2024cfa13f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.762393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a718da62ffa4afcacbe2214271d9bc4', 'request_id': 'req-7f82c9c9-fb41-413b-830d-2f2024cfa13f'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.772806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] The token cache is being invalidate because project 7a718da62ffa4afcacbe2214271d9bc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:23.773089 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.773713 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.775220 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1239/2445] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:23 2022] DELETE /identity/v3/projects/7a718da62ffa4afcacbe2214271d9bc4 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:23.779209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:23.789058 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab12950d-dbde-4de4-876f-531baddc25ae service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:23.843723 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:23.844379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:23.844661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:23.844929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] PATH_INFO: `/v3/users/bfd2963c66d04ae8a53db92b87191cae` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:23.846026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfd2963c66d04ae8a53db92b87191cae)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:23.847944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:23.894529 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfd2963c66d04ae8a53db92b87191cae'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.895306 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfd2963c66d04ae8a53db92b87191cae'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.902334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfd2963c66d04ae8a53db92b87191cae'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.914686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfd2963c66d04ae8a53db92b87191cae'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.915631 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfd2963c66d04ae8a53db92b87191cae'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:23.929763 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1207/2446] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:23 2022] DELETE /identity/v3/users/bfd2963c66d04ae8a53db92b87191cae => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:24.203823 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:24.205425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:24.224286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f82c9c9-fb41-413b-830d-2f2024cfa13f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:24.286691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:24.287366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:24.287576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:24.287839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] PATH_INFO: `/v3/projects/52df91f07d364b34a197b9b64cedc20a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:24.288992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] RBAC: Authorizing `identity:delete_project(project_id=52df91f07d364b34a197b9b64cedc20a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:24.290896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:24.343250 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52df91f07d364b34a197b9b64cedc20a', 'request_id': 'req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.351576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52df91f07d364b34a197b9b64cedc20a', 'request_id': 'req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.358098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52df91f07d364b34a197b9b64cedc20a', 'request_id': 'req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.370265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] The token cache is being invalidate because project 52df91f07d364b34a197b9b64cedc20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:24.370549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.371227 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.372796 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1240/2447] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:24 2022] DELETE /identity/v3/projects/52df91f07d364b34a197b9b64cedc20a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:24.380567 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:24.382050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:24.395143 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54418e80-887a-434d-bf3d-21e2f6038486 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:24.451298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:24.451889 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:24.452232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:24.452458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] PATH_INFO: `/v3/users/65f206a55ef74646aff5158af6a823ec` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:24.453581 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f206a55ef74646aff5158af6a823ec)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:24.458492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:24.485051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:24.529303 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f206a55ef74646aff5158af6a823ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.530048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f206a55ef74646aff5158af6a823ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.548112 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f206a55ef74646aff5158af6a823ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.563546 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f206a55ef74646aff5158af6a823ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.565425 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-d0b9da2b-f236-47a4-8f9a-7032149c2ce1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f206a55ef74646aff5158af6a823ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.586480 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1208/2448] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:24 2022] DELETE /identity/v3/users/65f206a55ef74646aff5158af6a823ec => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:24.872320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:24.872320 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:24.884024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d44e5-e3bf-4ac3-a542-6cffdfb6f23b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:24.910814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:24.943760 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:24.944406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:24.944692 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:24.944961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] PATH_INFO: `/v3/projects/67bd7b7119744a82a4cc590503e5affa` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:24.946056 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=67bd7b7119744a82a4cc590503e5affa)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:24.947932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:24.992049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67bd7b7119744a82a4cc590503e5affa', 'request_id': 'req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:24.998877 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67bd7b7119744a82a4cc590503e5affa', 'request_id': 'req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:25.005620 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67bd7b7119744a82a4cc590503e5affa', 'request_id': 'req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:25.015432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] The token cache is being invalidate because project 67bd7b7119744a82a4cc590503e5affa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:25.016272 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:25.016916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-0d0076d6-9664-4e0c-b6f4-de9f210ff4fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:25.018334 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1241/2449] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:24 2022] DELETE /identity/v3/projects/67bd7b7119744a82a4cc590503e5affa => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:25.024049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2bdb-97d9-4705-a872-6a6e6779b902 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.024273 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2bdb-97d9-4705-a872-6a6e6779b902 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.024569 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2bdb-97d9-4705-a872-6a6e6779b902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.045066 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-6eab2bdb-97d9-4705-a872-6a6e6779b902 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:25.046259 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-6eab2bdb-97d9-4705-a872-6a6e6779b902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:25.106725 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1209/2450] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:25.114471 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0612-d427-409d-9543-0e97b64cd8a1 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.114850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0612-d427-409d-9543-0e97b64cd8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.115066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31b0612-d427-409d-9543-0e97b64cd8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.135156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-e31b0612-d427-409d-9543-0e97b64cd8a1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:25.136369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-e31b0612-d427-409d-9543-0e97b64cd8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:25.144679 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1242/2451] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:25.148820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2bdb-97d9-4705-a872-6a6e6779b902 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.158239 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2bdb-97d9-4705-a872-6a6e6779b902 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.162733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.163324 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.163598 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.163867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.164550 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:25.164853 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.166433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.178581 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1210/2452] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:05:25.183018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0612-d427-409d-9543-0e97b64cd8a1 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.192845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31b0612-d427-409d-9543-0e97b64cd8a1 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.197389 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.197972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.198244 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.198509 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.199134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.201043 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.226922 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1243/2453] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:25.230952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.241196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42587b63-8cea-449d-80cd-554dff20f731 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.298147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.298672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.298949 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.299218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.299839 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.301787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.322577 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1211/2454] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:25.326636 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.337377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed4bf12-92c1-4c35-9307-7bff9858e656 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.341911 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.342505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.342781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.343050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.343687 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.345299 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.355506 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1244/2455] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:25.359208 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.369055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776b7ac-e0c2-4ec5-9f34-f2521922d07e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.373765 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cc55d-9608-4ce3-9e44-7f8d16ac99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.374340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cc55d-9608-4ce3-9e44-7f8d16ac99ca admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.374622 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cc55d-9608-4ce3-9e44-7f8d16ac99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.374893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cc55d-9608-4ce3-9e44-7f8d16ac99ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.375524 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cc55d-9608-4ce3-9e44-7f8d16ac99ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.377134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cc55d-9608-4ce3-9e44-7f8d16ac99ca admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.387858 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1212/2456] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:25.391967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.402506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f347189e-df58-42ff-ba0f-9151873559b4 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.407053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.407691 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.407970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.408240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] PATH_INFO: `/v3/projects/6287a85a08bf4a80a7a7ba1ea67b3687/users/ab0507677a2a4333b8d440f3925af8af/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.422839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=6287a85a08bf4a80a7a7ba1ea67b3687, user_id=ab0507677a2a4333b8d440f3925af8af, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.425551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.437776 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1245/2457] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:25 2022] PUT /identity/v3/projects/6287a85a08bf4a80a7a7ba1ea67b3687/users/ab0507677a2a4333b8d440f3925af8af/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:25.447940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d5fc4-f29f-474a-baa6-cbc122f2bf00 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.448256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d5fc4-f29f-474a-baa6-cbc122f2bf00 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.448558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d5fc4-f29f-474a-baa6-cbc122f2bf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.455539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:25.478482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-664d5fc4-f29f-474a-baa6-cbc122f2bf00 None None] MFA Rules not processed for user `ab0507677a2a4333b8d440f3925af8af`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:25.479676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-664d5fc4-f29f-474a-baa6-cbc122f2bf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:25.541980 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1213/2458] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:25.546464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.602897 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:25.608689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec4bb-9efb-4ee3-8cdb-203215deb6ce admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.614165 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.614761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.615073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.615340 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.616129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.618119 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.647424 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1246/2459] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:25.651213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d5fc4-f29f-474a-baa6-cbc122f2bf00 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.666222 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d5fc4-f29f-474a-baa6-cbc122f2bf00 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.721295 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.721962 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.722251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.722647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.723322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.725469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.749790 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1214/2460] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:25.754775 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.766118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0189a9de-5574-41f5-be35-6d6513c9c49e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.771772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.772419 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.772704 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.772980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.773624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.775603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.796809 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1247/2461] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:25.798078 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.808057 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb1589-6080-4613-8aa7-9c9c21a34d79 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.812609 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82616cc4-2c92-4c69-8811-e3c39158afd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.813227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82616cc4-2c92-4c69-8811-e3c39158afd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.813501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82616cc4-2c92-4c69-8811-e3c39158afd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.813769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82616cc4-2c92-4c69-8811-e3c39158afd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.814398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82616cc4-2c92-4c69-8811-e3c39158afd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.816574 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82616cc4-2c92-4c69-8811-e3c39158afd3 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.826817 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1215/2462] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:25.831075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:25.841096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682278e8-cb40-4814-a18c-b2425d693339 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:25.845566 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:25.846202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.846479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.846747 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] PATH_INFO: `/v3/projects/6d8dcc667f744f548df1d476d5bd4df9/users/750268da2b5449df9e18e4b511ba8d8c/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.861476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d8dcc667f744f548df1d476d5bd4df9, user_id=750268da2b5449df9e18e4b511ba8d8c, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:25.864722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:25.877594 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1248/2463] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:25 2022] PUT /identity/v3/projects/6d8dcc667f744f548df1d476d5bd4df9/users/750268da2b5449df9e18e4b511ba8d8c/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:25.889336 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187f664-1b2a-4a8c-baae-7febb0bd68ec None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:25.889647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187f664-1b2a-4a8c-baae-7febb0bd68ec None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:25.889921 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9187f664-1b2a-4a8c-baae-7febb0bd68ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:25.919128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-9187f664-1b2a-4a8c-baae-7febb0bd68ec None None] MFA Rules not processed for user `750268da2b5449df9e18e4b511ba8d8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:25.920345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-9187f664-1b2a-4a8c-baae-7febb0bd68ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:25.978729 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:25.982913 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1216/2464] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:25.986969 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:26.047560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc495e-d1e5-4823-99e9-d5646859ed8c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:26.052175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:26.052814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.053093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.053365 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.054006 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:26.055909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:26.081947 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1249/2465] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:25 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:26.085944 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187f664-1b2a-4a8c-baae-7febb0bd68ec None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:26.097492 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9187f664-1b2a-4a8c-baae-7febb0bd68ec None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:26.153362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:26.153987 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.154321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.154591 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.155365 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:26.157462 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:26.179737 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1217/2466] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:26 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:26.184117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:26.194302 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f7303-1e01-4d6b-a4d0-26aa424b2c5f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:26.198887 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:26.199475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.199758 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.200033 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.200721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:26.202327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:26.213287 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1250/2467] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:26.217162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:26.227352 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070342cd-72b4-423d-aae4-85cb9433f485 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:26.231843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:26.232498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.232777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.233048 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] PATH_INFO: `/v3/projects/1cdcceb2b24e41b6bca7d32e7be5a856/users/9ba68a453aeb48528e41387746dc9cee/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.247253 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdcceb2b24e41b6bca7d32e7be5a856, user_id=9ba68a453aeb48528e41387746dc9cee, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:26.249928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:26.262527 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1218/2468] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:26 2022] PUT /identity/v3/projects/1cdcceb2b24e41b6bca7d32e7be5a856/users/9ba68a453aeb48528e41387746dc9cee/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:26.266466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:26.277024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f68ce-2c34-4e85-a256-8c2be50c7e02 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:26.331036 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc634a-3986-4dcd-876f-a89f3caabd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:26.332239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc634a-3986-4dcd-876f-a89f3caabd8a admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.332546 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc634a-3986-4dcd-876f-a89f3caabd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.332815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc634a-3986-4dcd-876f-a89f3caabd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.333474 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc634a-3986-4dcd-876f-a89f3caabd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:26.335044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc634a-3986-4dcd-876f-a89f3caabd8a admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:26.344393 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1251/2469] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:26 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:26.348704 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:26.358721 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441963e-8ba8-49c1-a13f-e89bc1d61c07 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:26.363438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:26.364083 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.364396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.364668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] PATH_INFO: `/v3/projects/1cdcceb2b24e41b6bca7d32e7be5a856/users/9ba68a453aeb48528e41387746dc9cee/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.366552 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cdcceb2b24e41b6bca7d32e7be5a856, user_id=9ba68a453aeb48528e41387746dc9cee, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:26.369209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:26.379892 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1219/2470] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:26 2022] PUT /identity/v3/projects/1cdcceb2b24e41b6bca7d32e7be5a856/users/9ba68a453aeb48528e41387746dc9cee/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:26.389790 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.390073 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.390347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.417708 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] MFA Rules not processed for user `9ba68a453aeb48528e41387746dc9cee`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:26.418839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:26.422653 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:26.486947 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1252/2471] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:26.498457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:26.500227 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:26.557913 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24266940-9260-42d9-bf80-f3912190004e admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:26.561494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:26.562093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:26.562368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:26.562636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:26.619010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:26.620461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:26.629782 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1220/2472] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:26 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:28.262547 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:28.264150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:28.276866 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecd15a2-82dd-40ab-9e90-c8b757816b68 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:28.280535 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:28.281233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:28.281510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:28.281780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:28.352926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:28.354974 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:28.366651 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1253/2473] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:28.898793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:28.907410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8b5da-f602-4424-8062-12e4085e3a4f service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:28.911612 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd051dd-b1bd-465b-b2a2-8908554ef7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:28.912206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd051dd-b1bd-465b-b2a2-8908554ef7dd service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:28.912523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd051dd-b1bd-465b-b2a2-8908554ef7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:28.912793 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd051dd-b1bd-465b-b2a2-8908554ef7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:28.918423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:28.922797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd051dd-b1bd-465b-b2a2-8908554ef7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:28.924364 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd051dd-b1bd-465b-b2a2-8908554ef7dd service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:28.934177 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1221/2474] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:29.241428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:29.242821 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:29.306943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d8bea-a2d1-4bdd-8b8d-82f241dd3b13 service cinder] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:29.311507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:29.312149 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:29.312457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:29.312751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] PATH_INFO: `/v3/users/581a40a513e243d7a5fbd44f590fef95` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:29.313901 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=581a40a513e243d7a5fbd44f590fef95)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:29.316387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:29.356595 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:29.362015 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '581a40a513e243d7a5fbd44f590fef95'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.369526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '581a40a513e243d7a5fbd44f590fef95'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.376855 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '581a40a513e243d7a5fbd44f590fef95'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.389717 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '581a40a513e243d7a5fbd44f590fef95'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.390357 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '581a40a513e243d7a5fbd44f590fef95'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.396998 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1254/2475] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:29 2022] DELETE /identity/v3/users/581a40a513e243d7a5fbd44f590fef95 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:29.401580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:29.401864 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:29.402132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:29.424326 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:29.425556 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:29.484660 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1222/2476] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:29.494562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:29.496005 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:29.555423 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711854a1-58e9-4fe1-b53d-65915789d787 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:29.559082 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:29.559681 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:29.559962 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:29.560231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:29.569878 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:29.571306 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:29.580943 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1255/2477] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:29.861772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:29.864279 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:29.876557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46313cc6-17e9-4440-a567-17de80d125a1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:29.882490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:29.883146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:29.883410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:29.883683 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] PATH_INFO: `/v3/projects/92b75567eb6a4e98a4ee490dce493166` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:29.884830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b75567eb6a4e98a4ee490dce493166)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:29.886737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:29.932790 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b75567eb6a4e98a4ee490dce493166', 'request_id': 'req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.939855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b75567eb6a4e98a4ee490dce493166', 'request_id': 'req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.946155 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b75567eb6a4e98a4ee490dce493166', 'request_id': 'req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.956891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] The token cache is being invalidate because project 92b75567eb6a4e98a4ee490dce493166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:29.957176 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.957808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-205b95a9-65a1-4b20-ac23-7bf4d9e8fc0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:29.959264 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1223/2478] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:29 2022] DELETE /identity/v3/projects/92b75567eb6a4e98a4ee490dce493166 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:32.817947 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:32.819344 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:32.831940 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c564a907-3671-474a-b705-a70ae9cb1ab7 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:32.887308 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:32.887929 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:32.888203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:32.888508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] PATH_INFO: `/v3/users/ab0507677a2a4333b8d440f3925af8af` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:32.889640 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab0507677a2a4333b8d440f3925af8af)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:32.891539 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:32.935134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab0507677a2a4333b8d440f3925af8af'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:32.942368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab0507677a2a4333b8d440f3925af8af'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:32.949046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab0507677a2a4333b8d440f3925af8af'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:32.950334 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:32.963643 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab0507677a2a4333b8d440f3925af8af'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:32.964264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab0507677a2a4333b8d440f3925af8af'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:32.970030 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1256/2479] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:32 2022] DELETE /identity/v3/users/ab0507677a2a4333b8d440f3925af8af => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:32.974484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:32.974970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:32.975085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:32.994317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:32.995504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:33.051920 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1224/2480] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:33.061642 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:33.063007 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:33.118368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8137c275-3d9d-4b24-893b-0bf4e1598d3d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:33.122008 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:33.122603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:33.122880 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:33.123152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:33.132134 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:33.133580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:33.143392 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1257/2481] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:33.404091 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:33.405555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:33.418441 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae3e7ff-1f58-4ec0-9087-b3eb482bb57c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:33.423055 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:33.423685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:33.423959 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:33.424225 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] PATH_INFO: `/v3/projects/6287a85a08bf4a80a7a7ba1ea67b3687` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:33.426165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6287a85a08bf4a80a7a7ba1ea67b3687)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:33.427953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:33.441141 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:33.479682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6287a85a08bf4a80a7a7ba1ea67b3687', 'request_id': 'req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.487827 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6287a85a08bf4a80a7a7ba1ea67b3687', 'request_id': 'req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.493854 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6287a85a08bf4a80a7a7ba1ea67b3687', 'request_id': 'req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.505119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] The token cache is being invalidate because project 6287a85a08bf4a80a7a7ba1ea67b3687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:33.505401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.506036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.507499 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1225/2482] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:33 2022] DELETE /identity/v3/projects/6287a85a08bf4a80a7a7ba1ea67b3687 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:33.511431 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:33.521438 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bdb026-474e-4c81-97a5-406811f2baf3 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:33.579730 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:33.580496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:33.580773 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:33.581098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] PATH_INFO: `/v3/users/750268da2b5449df9e18e4b511ba8d8c` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:33.582928 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] RBAC: Authorizing `identity:delete_user(user_id=750268da2b5449df9e18e4b511ba8d8c)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:33.584946 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:33.639226 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '750268da2b5449df9e18e4b511ba8d8c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.646714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '750268da2b5449df9e18e4b511ba8d8c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.654482 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '750268da2b5449df9e18e4b511ba8d8c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.668544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '750268da2b5449df9e18e4b511ba8d8c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.669196 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '750268da2b5449df9e18e4b511ba8d8c'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:33.675614 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1258/2483] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:33 2022] DELETE /identity/v3/users/750268da2b5449df9e18e4b511ba8d8c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:33.967859 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:33.969331 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:33.983941 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf4765-6bdf-45a5-b4f4-f212f9a26f6c admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:34.041330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:34.041958 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.042232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.042498 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] PATH_INFO: `/v3/projects/6d8dcc667f744f548df1d476d5bd4df9` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.043582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d8dcc667f744f548df1d476d5bd4df9)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:34.045494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:34.090716 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d8dcc667f744f548df1d476d5bd4df9', 'request_id': 'req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.098196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d8dcc667f744f548df1d476d5bd4df9', 'request_id': 'req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.103795 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d8dcc667f744f548df1d476d5bd4df9', 'request_id': 'req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.115472 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] The token cache is being invalidate because project 6d8dcc667f744f548df1d476d5bd4df9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:34.115752 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.116399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.117833 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1226/2484] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:33 2022] DELETE /identity/v3/projects/6d8dcc667f744f548df1d476d5bd4df9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:34.121982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:34.123406 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:34.131470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:34.137526 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32a15e-e9c5-4c0f-9135-4c3794d95702 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:34.194075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:34.194812 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.195155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.195428 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] PATH_INFO: `/v3/users/9ba68a453aeb48528e41387746dc9cee` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.196571 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ba68a453aeb48528e41387746dc9cee)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:34.198481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:34.244796 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ba68a453aeb48528e41387746dc9cee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.251558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ba68a453aeb48528e41387746dc9cee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.258967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ba68a453aeb48528e41387746dc9cee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.271122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ba68a453aeb48528e41387746dc9cee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.271805 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-982bc6cf-47bf-45f5-be2e-1c49ce754142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ba68a453aeb48528e41387746dc9cee'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.277869 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1259/2485] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:34 2022] DELETE /identity/v3/users/9ba68a453aeb48528e41387746dc9cee => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:34.539661 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:34.541186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:34.554201 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6812e-daaa-4d16-a566-cdc46cf9ebd8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:34.583891 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:34.615763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:34.616351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.616630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.616896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] PATH_INFO: `/v3/projects/1cdcceb2b24e41b6bca7d32e7be5a856` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.617990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cdcceb2b24e41b6bca7d32e7be5a856)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:34.619867 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:34.664876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cdcceb2b24e41b6bca7d32e7be5a856', 'request_id': 'req-f62c4a98-cf36-43a3-aa87-ea859978444b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.671803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cdcceb2b24e41b6bca7d32e7be5a856', 'request_id': 'req-f62c4a98-cf36-43a3-aa87-ea859978444b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.678183 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cdcceb2b24e41b6bca7d32e7be5a856', 'request_id': 'req-f62c4a98-cf36-43a3-aa87-ea859978444b'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.689132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] The token cache is being invalidate because project 1cdcceb2b24e41b6bca7d32e7be5a856 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:34.689411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.690028 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-f62c4a98-cf36-43a3-aa87-ea859978444b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:34.692012 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1227/2486] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:34 2022] DELETE /identity/v3/projects/1cdcceb2b24e41b6bca7d32e7be5a856 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:34.697753 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b87d3-d624-41b5-ab57-9ef5d487d5cb None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.698110 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b87d3-d624-41b5-ab57-9ef5d487d5cb None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.698548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b87d3-d624-41b5-ab57-9ef5d487d5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.718188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-471b87d3-d624-41b5-ab57-9ef5d487d5cb None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:34.719377 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-471b87d3-d624-41b5-ab57-9ef5d487d5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:34.776969 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1260/2487] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:34.785280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92f88-8b8e-456d-adc2-0e0c9ac17249 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.785563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92f88-8b8e-456d-adc2-0e0c9ac17249 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.785841 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa92f88-8b8e-456d-adc2-0e0c9ac17249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.806406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-4aa92f88-8b8e-456d-adc2-0e0c9ac17249 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:34.807594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4aa92f88-8b8e-456d-adc2-0e0c9ac17249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:34.815772 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1228/2488] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:34.819727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b87d3-d624-41b5-ab57-9ef5d487d5cb None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:34.829364 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b87d3-d624-41b5-ab57-9ef5d487d5cb None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:34.833973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:34.834562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.834836 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.835104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.835741 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:34.836064 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:34.837742 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:34.850190 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1261/2489] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:05:34.856377 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa92f88-8b8e-456d-adc2-0e0c9ac17249 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:34.866445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa92f88-8b8e-456d-adc2-0e0c9ac17249 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:34.870969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:34.871565 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.871835 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.872123 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.872776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:34.874653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:34.904349 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1229/2490] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:34 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:34.908637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:34.919683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0491db-b5e8-4697-b63f-2b12cf8483c6 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:34.924813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:34.980854 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:34.981470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:34.981751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:34.982024 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:34.982662 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:34.984649 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:35.006727 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1262/2491] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:34 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:05:35.010836 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:35.022388 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25fa05c-dff4-48f9-9826-f27e17004a26 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:35.027881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:35.028763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:35.029032 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:35.029307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:35.030138 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:35.031900 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:35.043563 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1230/2492] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:35.047621 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:35.057873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278593af-cee0-4943-b5af-f405d39607b7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:35.062494 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c477315-9176-4e7f-aaf2-64f0342dea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:35.063084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477315-9176-4e7f-aaf2-64f0342dea31 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:35.063361 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477315-9176-4e7f-aaf2-64f0342dea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:35.063632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477315-9176-4e7f-aaf2-64f0342dea31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:35.064269 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c477315-9176-4e7f-aaf2-64f0342dea31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:35.065906 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c477315-9176-4e7f-aaf2-64f0342dea31 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:35.077854 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1263/2493] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:35.082728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:35.094099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f9e3d-f554-4801-9bcb-59c2bd942df1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:35.099490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:35.100146 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:35.100458 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:35.100756 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] PATH_INFO: `/v3/projects/27f59bb062964c75bbdc8717cbcdc6bd/users/e396d912644849cb82bf10d5cfcd6481/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:35.116153 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f59bb062964c75bbdc8717cbcdc6bd, user_id=e396d912644849cb82bf10d5cfcd6481, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:35.118974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:35.131686 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1231/2494] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:35 2022] PUT /identity/v3/projects/27f59bb062964c75bbdc8717cbcdc6bd/users/e396d912644849cb82bf10d5cfcd6481/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:35.141674 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:35.142142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:35.142268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:35.169532 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] MFA Rules not processed for user `e396d912644849cb82bf10d5cfcd6481`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:35.170705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:35.231888 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1264/2495] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:35.242741 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:35.244451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:35.300789 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152a727f-e2d8-4b3a-bc43-7dbc073d4ab7 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:35.304417 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f735570-a97c-4bf1-99c6-ba7b1a78bc96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:35.305022 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f735570-a97c-4bf1-99c6-ba7b1a78bc96 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:35.305305 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f735570-a97c-4bf1-99c6-ba7b1a78bc96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:35.305595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f735570-a97c-4bf1-99c6-ba7b1a78bc96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:35.313696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f735570-a97c-4bf1-99c6-ba7b1a78bc96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:35.315111 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f735570-a97c-4bf1-99c6-ba7b1a78bc96 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:35.320029 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:35.325200 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1232/2496] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:39.149162 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:39.150622 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:39.214014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc9afd-83af-4c18-9e93-40e0504da7c6 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:39.218479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:39.219091 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:39.219368 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:39.219635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] PATH_INFO: `/v3/users/452665f2613f4b9b98fc2d2d924c8f1e` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:39.220750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=452665f2613f4b9b98fc2d2d924c8f1e)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:39.223152 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:39.243478 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:39.269701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452665f2613f4b9b98fc2d2d924c8f1e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.277104 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452665f2613f4b9b98fc2d2d924c8f1e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.285591 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452665f2613f4b9b98fc2d2d924c8f1e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.297447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452665f2613f4b9b98fc2d2d924c8f1e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.298070 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452665f2613f4b9b98fc2d2d924c8f1e'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.304383 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1265/2497] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:39 2022] DELETE /identity/v3/users/452665f2613f4b9b98fc2d2d924c8f1e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:39.308782 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:39.309281 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:39.309379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:39.327907 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:39.329121 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:39.385740 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1233/2498] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:39.395802 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:39.397204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:39.455481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa68d1ff-c2fc-485b-b027-9fc8447170b7 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:39.459633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:39.460234 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:39.460541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:39.460806 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:39.469688 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:39.471094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:39.481294 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1266/2499] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:39.759165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:39.760627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:39.773774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28c77e0-749b-4192-a467-a5c220f48a7d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:39.778469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:39.779199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:39.779380 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:39.779794 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] PATH_INFO: `/v3/projects/d603a305270945de898af14beb1734a0` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:39.780807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d603a305270945de898af14beb1734a0)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:39.782813 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:39.835080 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd603a305270945de898af14beb1734a0', 'request_id': 'req-81543b7f-79bd-4347-8f27-d78eaa977580'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.843280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd603a305270945de898af14beb1734a0', 'request_id': 'req-81543b7f-79bd-4347-8f27-d78eaa977580'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.849478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd603a305270945de898af14beb1734a0', 'request_id': 'req-81543b7f-79bd-4347-8f27-d78eaa977580'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.861024 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] The token cache is being invalidate because project d603a305270945de898af14beb1734a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:39.861419 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.862122 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.863717 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1234/2500] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:39 2022] DELETE /identity/v3/projects/d603a305270945de898af14beb1734a0 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:39.867817 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:39.877287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb026659-9465-4caf-ac31-a342cafad56e service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:39.936224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:39.936952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:39.937229 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:39.937500 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] PATH_INFO: `/v3/users/bda1b4bba2694b4e8f8ef8bc0ceaf3a0` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:39.938626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda1b4bba2694b4e8f8ef8bc0ceaf3a0)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:39.940624 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:39.982648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:39.988705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda1b4bba2694b4e8f8ef8bc0ceaf3a0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:39.996840 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda1b4bba2694b4e8f8ef8bc0ceaf3a0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.005603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda1b4bba2694b4e8f8ef8bc0ceaf3a0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.017714 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda1b4bba2694b4e8f8ef8bc0ceaf3a0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.018362 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-fd285c84-7b6a-4a8b-9fe7-e4e66afd708d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda1b4bba2694b4e8f8ef8bc0ceaf3a0'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.028620 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1267/2501] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:39 2022] DELETE /identity/v3/users/bda1b4bba2694b4e8f8ef8bc0ceaf3a0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:40.309534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:40.315366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:40.331875 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81543b7f-79bd-4347-8f27-d78eaa977580 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:40.389882 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:40.390528 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.390822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.391095 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] PATH_INFO: `/v3/projects/8f6e3f870a864895822901aa5a9a7b35` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.392209 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f6e3f870a864895822901aa5a9a7b35)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:40.394211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:40.423737 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:40.443195 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f6e3f870a864895822901aa5a9a7b35', 'request_id': 'req-5248506a-b0b7-4f83-a729-3626289e45f1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.450914 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f6e3f870a864895822901aa5a9a7b35', 'request_id': 'req-5248506a-b0b7-4f83-a729-3626289e45f1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.456924 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f6e3f870a864895822901aa5a9a7b35', 'request_id': 'req-5248506a-b0b7-4f83-a729-3626289e45f1'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.468383 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] The token cache is being invalidate because project 8f6e3f870a864895822901aa5a9a7b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:40.468693 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.469315 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-5248506a-b0b7-4f83-a729-3626289e45f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:40.471047 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1235/2502] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:40 2022] DELETE /identity/v3/projects/8f6e3f870a864895822901aa5a9a7b35 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:40.475693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a41ea-fbdf-4d36-a0af-e6808cf20544 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.476171 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a41ea-fbdf-4d36-a0af-e6808cf20544 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.476273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7a41ea-fbdf-4d36-a0af-e6808cf20544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.497776 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fe7a41ea-fbdf-4d36-a0af-e6808cf20544 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:40.498976 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fe7a41ea-fbdf-4d36-a0af-e6808cf20544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:40.559691 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1268/2503] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:40.568911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c114e-ff54-4a63-b1a8-7d108ceb0de3 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.569196 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c114e-ff54-4a63-b1a8-7d108ceb0de3 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.569584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31c114e-ff54-4a63-b1a8-7d108ceb0de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.599050 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e31c114e-ff54-4a63-b1a8-7d108ceb0de3 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:40.600379 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e31c114e-ff54-4a63-b1a8-7d108ceb0de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:40.609270 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1236/2504] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:40.614314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a41ea-fbdf-4d36-a0af-e6808cf20544 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:40.624267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7a41ea-fbdf-4d36-a0af-e6808cf20544 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:40.629210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:40.629803 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.630075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.630337 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.630973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81372) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:40.631271 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:40.632902 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:40.644687 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1269/2505] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:05:40.649139 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c114e-ff54-4a63-b1a8-7d108ceb0de3 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:40.658206 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31c114e-ff54-4a63-b1a8-7d108ceb0de3 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:40.663234 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:40.663826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.664101 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.664410 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.665031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:40.666898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:40.695815 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1237/2506] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:40 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:40.700175 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:40.710068 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdc7881-58b8-4c03-944a-dd6570f8c3f0 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:40.772366 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:40.773014 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.773290 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.773558 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.774221 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:40.776084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:40.804510 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1270/2507] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:40 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:40.805826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:40.816638 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8d7cd-092c-4d0e-9ef1-dc8ce09faaf1 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:40.821608 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:40.822293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.822629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.822961 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.823694 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:40.830340 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:40.841666 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1238/2508] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:40.845648 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:40.856680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51971d4-98bd-4ac3-84e0-d8472a7a6d56 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:40.860630 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83dde9-3209-4c80-aab4-dcf184018886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:40.861219 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83dde9-3209-4c80-aab4-dcf184018886 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.861496 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83dde9-3209-4c80-aab4-dcf184018886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.861768 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83dde9-3209-4c80-aab4-dcf184018886 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.862404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83dde9-3209-4c80-aab4-dcf184018886 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:40.864703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83dde9-3209-4c80-aab4-dcf184018886 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:40.878796 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1271/2509] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:40.880052 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:40.889919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730b0c5-13d9-455e-89b2-cbd2d57a8fb2 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:40.895010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:40.895658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.895931 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.896204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] PATH_INFO: `/v3/projects/e288476b64d545fbaba2148f37cb04cd/users/89311129a2864f46b84aaa4d60a7bd72/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.912772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e288476b64d545fbaba2148f37cb04cd, user_id=89311129a2864f46b84aaa4d60a7bd72, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:40.915416 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:40.929811 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1239/2510] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:40 2022] PUT /identity/v3/projects/e288476b64d545fbaba2148f37cb04cd/users/89311129a2864f46b84aaa4d60a7bd72/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:40.940615 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62313-800c-4dbf-b69b-e80ac3e4f25d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:40.940896 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62313-800c-4dbf-b69b-e80ac3e4f25d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:40.941195 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e62313-800c-4dbf-b69b-e80ac3e4f25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:40.969597 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-23e62313-800c-4dbf-b69b-e80ac3e4f25d None None] MFA Rules not processed for user `89311129a2864f46b84aaa4d60a7bd72`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:40.970957 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-23e62313-800c-4dbf-b69b-e80ac3e4f25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:40.983701 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:41.038888 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1272/2511] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:41.043554 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:41.106594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a73c30-c868-49bf-a405-439754b679ee admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:41.111433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:41.112036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.112338 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.112616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.113301 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:41.115240 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:41.143395 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1240/2512] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:41 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:41.148132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e62313-800c-4dbf-b69b-e80ac3e4f25d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:41.158813 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e62313-800c-4dbf-b69b-e80ac3e4f25d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:41.215133 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:41.215743 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.216026 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.216314 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] PATH_INFO: `/v3/users` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.216978 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:41.218913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:41.241291 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1273/2513] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:41 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:41.245378 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:41.255917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ea3199-dea7-43a6-b152-736ccb8fa2d5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:41.260510 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:41.261106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.261384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.261655 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.262289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:41.263863 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:41.278955 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1241/2514] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:41.280001 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:41.290445 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e95030-2da7-4298-938c-422479f828fb admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:41.295166 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:41.295837 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.296137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.296437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] PATH_INFO: `/v3/projects/d23e37afed6c4ded80b7f264e4a7b961/users/d4d67d8002374dbe8a09ac26e2ff7c31/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.311121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23e37afed6c4ded80b7f264e4a7b961, user_id=d4d67d8002374dbe8a09ac26e2ff7c31, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:41.313787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:41.327318 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1274/2515] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:41 2022] PUT /identity/v3/projects/d23e37afed6c4ded80b7f264e4a7b961/users/d4d67d8002374dbe8a09ac26e2ff7c31/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:41.331298 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:41.345347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc81d82c-1184-462d-bdce-fda8653b7187 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:41.362719 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:41.403772 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86320fcd-5a7a-4958-bd4c-0194ab3c6d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:41.404414 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320fcd-5a7a-4958-bd4c-0194ab3c6d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.404696 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320fcd-5a7a-4958-bd4c-0194ab3c6d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.404966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86320fcd-5a7a-4958-bd4c-0194ab3c6d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.405602 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86320fcd-5a7a-4958-bd4c-0194ab3c6d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:41.407204 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86320fcd-5a7a-4958-bd4c-0194ab3c6d41 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:41.417566 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1242/2516] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:41.421723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:41.431846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2bf79f-03b2-4eae-98cb-d4510003f81f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:41.436456 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:41.437093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.437370 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.437639 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] PATH_INFO: `/v3/projects/d23e37afed6c4ded80b7f264e4a7b961/users/d4d67d8002374dbe8a09ac26e2ff7c31/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.439529 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23e37afed6c4ded80b7f264e4a7b961, user_id=d4d67d8002374dbe8a09ac26e2ff7c31, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:41.442723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:41.452817 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1275/2517] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:41 2022] PUT /identity/v3/projects/d23e37afed6c4ded80b7f264e4a7b961/users/d4d67d8002374dbe8a09ac26e2ff7c31/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:41.463520 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977aba8-e100-43d2-9a2a-c8fafbc6cb8d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.463804 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977aba8-e100-43d2-9a2a-c8fafbc6cb8d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.464104 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2977aba8-e100-43d2-9a2a-c8fafbc6cb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.495063 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-2977aba8-e100-43d2-9a2a-c8fafbc6cb8d None None] MFA Rules not processed for user `d4d67d8002374dbe8a09ac26e2ff7c31`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:41.496277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-2977aba8-e100-43d2-9a2a-c8fafbc6cb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:41.568321 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1243/2518] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:41.579508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:41.581304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:41.644373 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6ee353-15ed-4649-bdeb-dc26581ef7a8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:41.648586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14100f-5c3b-4f85-b8d0-379216413d31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:41.649505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14100f-5c3b-4f85-b8d0-379216413d31 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:41.649781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14100f-5c3b-4f85-b8d0-379216413d31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:41.650050 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14100f-5c3b-4f85-b8d0-379216413d31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:41.660075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14100f-5c3b-4f85-b8d0-379216413d31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:41.661564 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14100f-5c3b-4f85-b8d0-379216413d31 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:41.672021 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1276/2519] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:42.071177 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977aba8-e100-43d2-9a2a-c8fafbc6cb8d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:42.135975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2977aba8-e100-43d2-9a2a-c8fafbc6cb8d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:42.140733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:42.141404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:42.141708 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:42.142017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] PATH_INFO: `/v3/users/89311129a2864f46b84aaa4d60a7bd72` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:42.143247 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89311129a2864f46b84aaa4d60a7bd72)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:42.145358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:42.193010 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89311129a2864f46b84aaa4d60a7bd72'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.193667 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89311129a2864f46b84aaa4d60a7bd72'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.194896 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:42.202457 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89311129a2864f46b84aaa4d60a7bd72'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.215358 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89311129a2864f46b84aaa4d60a7bd72'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.215673 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89311129a2864f46b84aaa4d60a7bd72'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.228756 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1244/2520] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:42 2022] DELETE /identity/v3/users/89311129a2864f46b84aaa4d60a7bd72 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:42.233259 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:42.233608 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:42.233948 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:42.253958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:42.255139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:42.258994 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:42.313832 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1277/2521] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:42.323421 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:42.324820 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:42.383189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73c49f0-0a73-4a82-84c4-78310b7aa714 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:42.386989 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:42.387699 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:42.387974 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:42.388238 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:42.398044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:42.399490 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:42.408779 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1245/2522] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:42.671873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:42.673375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:42.685249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156cdf3b-9ceb-4e46-acdb-f1c8d3c0897f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:42.700746 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:42.701393 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:42.701672 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:42.701945 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] PATH_INFO: `/v3/projects/e288476b64d545fbaba2148f37cb04cd` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:42.703121 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e288476b64d545fbaba2148f37cb04cd)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:42.705074 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:42.753265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e288476b64d545fbaba2148f37cb04cd', 'request_id': 'req-a7bbe51b-2479-4412-a8ca-4e2a835aa994'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.760628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e288476b64d545fbaba2148f37cb04cd', 'request_id': 'req-a7bbe51b-2479-4412-a8ca-4e2a835aa994'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.766658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e288476b64d545fbaba2148f37cb04cd', 'request_id': 'req-a7bbe51b-2479-4412-a8ca-4e2a835aa994'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.777258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] The token cache is being invalidate because project e288476b64d545fbaba2148f37cb04cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:42.777534 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.778144 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.779572 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1278/2523] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:42 2022] DELETE /identity/v3/projects/e288476b64d545fbaba2148f37cb04cd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:42.783505 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:42.793079 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbbd95-6f72-4cd5-a326-bc5e5bc761c9 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:42.847978 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:42.848634 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:42.848906 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:42.849170 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] PATH_INFO: `/v3/users/d4d67d8002374dbe8a09ac26e2ff7c31` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:42.850256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4d67d8002374dbe8a09ac26e2ff7c31)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:42.852150 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:42.895866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4d67d8002374dbe8a09ac26e2ff7c31'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.896522 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4d67d8002374dbe8a09ac26e2ff7c31'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.903207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4d67d8002374dbe8a09ac26e2ff7c31'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.913065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:42.915575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4d67d8002374dbe8a09ac26e2ff7c31'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.916627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-97f9367d-70d3-4e50-be6c-3ebe38eb8b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4d67d8002374dbe8a09ac26e2ff7c31'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:42.929189 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1246/2524] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:42 2022] DELETE /identity/v3/users/d4d67d8002374dbe8a09ac26e2ff7c31 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:43.206164 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.207852 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:43.229744 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bbe51b-2479-4412-a8ca-4e2a835aa994 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.290177 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.290724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.290998 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.291264 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] PATH_INFO: `/v3/projects/d23e37afed6c4ded80b7f264e4a7b961` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.292390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23e37afed6c4ded80b7f264e4a7b961)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.294286 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:43.331749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:43.344276 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23e37afed6c4ded80b7f264e4a7b961', 'request_id': 'req-314d2c8c-6520-4859-a4e6-c76be38d56c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:43.353185 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23e37afed6c4ded80b7f264e4a7b961', 'request_id': 'req-314d2c8c-6520-4859-a4e6-c76be38d56c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:43.359543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23e37afed6c4ded80b7f264e4a7b961', 'request_id': 'req-314d2c8c-6520-4859-a4e6-c76be38d56c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:43.370683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] The token cache is being invalidate because project d23e37afed6c4ded80b7f264e4a7b961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:43.370967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:43.371603 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-314d2c8c-6520-4859-a4e6-c76be38d56c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:43.373481 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1279/2525] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:43 2022] DELETE /identity/v3/projects/d23e37afed6c4ded80b7f264e4a7b961 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:43.378035 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c650-ed84-41d3-96b8-0b91fb144a04 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.378323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c650-ed84-41d3-96b8-0b91fb144a04 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.379289 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c650-ed84-41d3-96b8-0b91fb144a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.400232 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e046c650-ed84-41d3-96b8-0b91fb144a04 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:43.401451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e046c650-ed84-41d3-96b8-0b91fb144a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:43.458687 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1247/2526] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:43.466536 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdfaee-8dd3-4c21-a5cd-021c3eabaff4 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.466845 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdfaee-8dd3-4c21-a5cd-021c3eabaff4 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.467142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cdfaee-8dd3-4c21-a5cd-021c3eabaff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.486563 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-74cdfaee-8dd3-4c21-a5cd-021c3eabaff4 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:43.488341 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-74cdfaee-8dd3-4c21-a5cd-021c3eabaff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:43.496025 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1280/2527] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:43.500197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c650-ed84-41d3-96b8-0b91fb144a04 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.510335 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c650-ed84-41d3-96b8-0b91fb144a04 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.514815 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.515404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.515682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.515951 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.516616 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:43.516919 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.518479 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:43.529717 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1248/2528] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:05:43.533810 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdfaee-8dd3-4c21-a5cd-021c3eabaff4 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.543481 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cdfaee-8dd3-4c21-a5cd-021c3eabaff4 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.548094 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.548729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.549012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.549287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.549912 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.552466 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:43.583156 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1281/2529] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:43 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:43.587411 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.599213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f944d-f5cc-4a4b-a8c6-04cc51594914 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.656384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.656991 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.657274 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.657547 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.658179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.660636 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:43.681478 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1249/2530] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:43 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:43.685619 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.695975 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d290769-e1bb-4b10-8ed0-d7212bdc4968 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.701000 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.701607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.701916 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.702188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.702861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.704602 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:43.718264 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1282/2531] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:43.719451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.730084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7820ac0-8c8f-4ab4-bd31-2d0342bd33ce admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.734849 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821908c1-b2f5-40f7-9574-c772cac5d327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.735453 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821908c1-b2f5-40f7-9574-c772cac5d327 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.735733 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821908c1-b2f5-40f7-9574-c772cac5d327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.736004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821908c1-b2f5-40f7-9574-c772cac5d327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.736678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821908c1-b2f5-40f7-9574-c772cac5d327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.739000 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821908c1-b2f5-40f7-9574-c772cac5d327 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:43.751289 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1250/2532] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:43.755723 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.765721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b254d-36b1-475c-a031-c95189d03d14 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.770319 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.770963 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.771242 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.771518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] PATH_INFO: `/v3/projects/fd28572195764b14888e504bee5960d2/users/562443cdd3ec4f41aa88389041bd2e25/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.786303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd28572195764b14888e504bee5960d2, user_id=562443cdd3ec4f41aa88389041bd2e25, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.789027 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:43.802177 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1283/2533] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:43 2022] PUT /identity/v3/projects/fd28572195764b14888e504bee5960d2/users/562443cdd3ec4f41aa88389041bd2e25/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:43.812911 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afc4a-6be2-4781-9534-0cb56035ed36 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.813197 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afc4a-6be2-4781-9534-0cb56035ed36 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.813460 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3afc4a-6be2-4781-9534-0cb56035ed36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.840117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-5b3afc4a-6be2-4781-9534-0cb56035ed36 None None] MFA Rules not processed for user `562443cdd3ec4f41aa88389041bd2e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:43.841344 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-5b3afc4a-6be2-4781-9534-0cb56035ed36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:43.881630 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:43.905590 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1251/2534] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:43.910804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:43.969698 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92412a63-0286-4eaa-94a3-753217b802a1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:43.974801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:43.975386 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:43.975659 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:43.975926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:43.976562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:43.978444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.004006 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1284/2535] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:43 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:44.007910 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afc4a-6be2-4781-9534-0cb56035ed36 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.018245 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3afc4a-6be2-4781-9534-0cb56035ed36 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.073580 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.074120 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.074401 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.074676 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.075307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.077271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.098806 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1252/2536] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:44 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:44.102835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.113236 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b65a5-71aa-4f1b-a4fa-35b867cb967f admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.117785 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.118382 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.118661 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.118938 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.119575 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.121206 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.131834 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1285/2537] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:44.135541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.145626 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e41e6c-ded5-4cd6-86ba-bbd2d30ab8e8 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.150186 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02cd28-e910-48df-880f-fe76a66f24f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.150767 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02cd28-e910-48df-880f-fe76a66f24f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.151049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02cd28-e910-48df-880f-fe76a66f24f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.151321 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02cd28-e910-48df-880f-fe76a66f24f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.151953 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02cd28-e910-48df-880f-fe76a66f24f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.153549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02cd28-e910-48df-880f-fe76a66f24f9 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.163913 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1253/2538] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:44.167667 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.177724 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8414f23-83cb-44ea-99e8-ab91487c6a21 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.182267 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.182900 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.183179 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.183451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] PATH_INFO: `/v3/projects/db74a41ac4094f90b78c550eb423ba2a/users/2d3cf95b93ec49b78fe0c2cda20076e6/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.197517 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db74a41ac4094f90b78c550eb423ba2a, user_id=2d3cf95b93ec49b78fe0c2cda20076e6, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.200168 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.213381 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1286/2539] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:44 2022] PUT /identity/v3/projects/db74a41ac4094f90b78c550eb423ba2a/users/2d3cf95b93ec49b78fe0c2cda20076e6/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:44.225026 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee36f-dd22-4594-a85e-20ed32ae298f None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.225311 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee36f-dd22-4594-a85e-20ed32ae298f None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.225585 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ee36f-dd22-4594-a85e-20ed32ae298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.252797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-378ee36f-dd22-4594-a85e-20ed32ae298f None None] MFA Rules not processed for user `2d3cf95b93ec49b78fe0c2cda20076e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:44.254558 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-378ee36f-dd22-4594-a85e-20ed32ae298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:44.322740 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1254/2540] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:44 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:44.327864 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.339804 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:44.398371 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9deb18-aefe-4df4-bc70-c335533f48c3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.403065 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.403633 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.403913 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.404186 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.404888 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.406873 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.433038 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1287/2541] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:44 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:44.437857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee36f-dd22-4594-a85e-20ed32ae298f None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.450051 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ee36f-dd22-4594-a85e-20ed32ae298f None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.499355 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:44.510898 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.511629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.512578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.512857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.513675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.516343 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.541250 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1255/2542] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:44 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:44.542453 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.552685 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357d7ec-8045-4c57-b688-7783389014e8 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.557273 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.557869 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.558150 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.558421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.559062 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.560689 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.571645 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1288/2543] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:44.575946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.586203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94f742f-6ba4-4c19-8ba8-c3fb8c7f96ed admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.590778 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.591433 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.591713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.592004 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] PATH_INFO: `/v3/projects/d6f9964b36d64f998910993e6ca17c61/users/1729ddadad72404fad951ba192b4f80a/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.609619 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f9964b36d64f998910993e6ca17c61, user_id=1729ddadad72404fad951ba192b4f80a, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.612265 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.624992 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1256/2544] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:44 2022] PUT /identity/v3/projects/d6f9964b36d64f998910993e6ca17c61/users/1729ddadad72404fad951ba192b4f80a/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:44.629556 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.639410 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a86108-b7d7-4611-8e51-480560739829 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.694327 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d22f3b-025f-4fc7-9907-317d18831ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.694941 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22f3b-025f-4fc7-9907-317d18831ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.695251 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22f3b-025f-4fc7-9907-317d18831ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.695472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d22f3b-025f-4fc7-9907-317d18831ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.696182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d22f3b-025f-4fc7-9907-317d18831ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.697794 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d22f3b-025f-4fc7-9907-317d18831ac9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.707251 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1289/2545] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:44.711549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.721769 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8b22d-2715-4300-b9d9-c224f6738aee admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.726322 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.726960 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.727235 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.727501 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] PATH_INFO: `/v3/projects/d6f9964b36d64f998910993e6ca17c61/users/1729ddadad72404fad951ba192b4f80a/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.729426 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f9964b36d64f998910993e6ca17c61, user_id=1729ddadad72404fad951ba192b4f80a, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.732009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.742762 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1257/2546] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:44 2022] PUT /identity/v3/projects/d6f9964b36d64f998910993e6ca17c61/users/1729ddadad72404fad951ba192b4f80a/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:44.752903 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.753183 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.754096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.782194 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] MFA Rules not processed for user `1729ddadad72404fad951ba192b4f80a`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:44.783387 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:44.851136 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1290/2547] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:44.861666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:44.863918 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:44.919637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a13d62-aca6-455b-9630-26a835c33d84 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:44.923261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:44.923857 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:44.924137 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:44.924445 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:44.933807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:44.935233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:44.945184 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1258/2548] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:45.427628 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:45.429105 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:45.435727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:45.436346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:45.445224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf286e-393c-4ab3-b4bb-022d8bafbf62 service cinder] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:45.449168 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543a346-9457-4a78-ab0f-b8ae962b692e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:45.449762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543a346-9457-4a78-ab0f-b8ae962b692e service cinder] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:45.450034 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543a346-9457-4a78-ab0f-b8ae962b692e service cinder] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:45.450302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543a346-9457-4a78-ab0f-b8ae962b692e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:45.497432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7e57d-c35f-4d89-96d9-284b4ec5c0ed None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:45.501980 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:45.502607 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:45.502884 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:45.503156 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] PATH_INFO: `/v3/users/e396d912644849cb82bf10d5cfcd6481` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:45.504598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e396d912644849cb82bf10d5cfcd6481)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:45.506543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:45.508406 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543a346-9457-4a78-ab0f-b8ae962b692e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:45.509860 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543a346-9457-4a78-ab0f-b8ae962b692e service cinder] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:45.519580 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1259/2549] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:05:45.551645 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e396d912644849cb82bf10d5cfcd6481'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:45.558592 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e396d912644849cb82bf10d5cfcd6481'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:45.567093 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e396d912644849cb82bf10d5cfcd6481'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:45.579199 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e396d912644849cb82bf10d5cfcd6481'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:45.579934 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e396d912644849cb82bf10d5cfcd6481'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:45.586703 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1291/2550] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:45 2022] DELETE /identity/v3/users/e396d912644849cb82bf10d5cfcd6481 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:45.591096 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:45.591351 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:45.591624 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:45.612230 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:45.613586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:45.657116 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:45.679582 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1260/2551] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:45.689518 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:45.691173 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:45.751188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b687faef-da3a-4421-b3bf-903ef9b9eaaf admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:45.756061 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:45.756716 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:45.756992 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:45.757265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:45.766781 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:45.768205 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:45.778425 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1292/2552] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:46.047277 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:46.048702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:46.061583 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc564f1e-77c5-4cf6-8f7e-31f41f3113f8 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:46.066976 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:46.067722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:46.068098 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:46.068469 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] PATH_INFO: `/v3/projects/27f59bb062964c75bbdc8717cbcdc6bd` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:46.069599 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f59bb062964c75bbdc8717cbcdc6bd)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:46.073084 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:46.119171 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f59bb062964c75bbdc8717cbcdc6bd', 'request_id': 'req-daf3aa53-bbc2-4324-92a1-497b24bfeded'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:46.125675 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f59bb062964c75bbdc8717cbcdc6bd', 'request_id': 'req-daf3aa53-bbc2-4324-92a1-497b24bfeded'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:46.131302 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f59bb062964c75bbdc8717cbcdc6bd', 'request_id': 'req-daf3aa53-bbc2-4324-92a1-497b24bfeded'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:46.141884 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] The token cache is being invalidate because project 27f59bb062964c75bbdc8717cbcdc6bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81371) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:46.142165 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:46.142776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:46.144760 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1261/2553] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:46 2022] DELETE /identity/v3/projects/27f59bb062964c75bbdc8717cbcdc6bd => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:49.554989 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:49.556413 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:49.586258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:49.614883 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cad721-cbcf-49fa-beee-e3635a0b6a71 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:49.661053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b4aa26-ae8f-42e4-92fa-fbace3e53358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:49.661646 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4aa26-ae8f-42e4-92fa-fbace3e53358 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:49.661920 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4aa26-ae8f-42e4-92fa-fbace3e53358 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:49.662188 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4aa26-ae8f-42e4-92fa-fbace3e53358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:49.663872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b9b4aa26-ae8f-42e4-92fa-fbace3e53358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:49.719549 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4aa26-ae8f-42e4-92fa-fbace3e53358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:49.720995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b4aa26-ae8f-42e4-92fa-fbace3e53358 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:49.730150 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1293/2554] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:05:51.766929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:51.768443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:51.831069 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3aa53-bbc2-4324-92a1-497b24bfeded admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:51.835557 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:51.836179 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:51.836493 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:51.836763 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] PATH_INFO: `/v3/users/562443cdd3ec4f41aa88389041bd2e25` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:51.837894 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=562443cdd3ec4f41aa88389041bd2e25)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:51.839776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:51.885046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '562443cdd3ec4f41aa88389041bd2e25'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:51.885689 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '562443cdd3ec4f41aa88389041bd2e25'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:51.892682 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '562443cdd3ec4f41aa88389041bd2e25'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:51.903049 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:51.905449 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '562443cdd3ec4f41aa88389041bd2e25'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:51.906564 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '562443cdd3ec4f41aa88389041bd2e25'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:51.919977 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1262/2555] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:51 2022] DELETE /identity/v3/users/562443cdd3ec4f41aa88389041bd2e25 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:51.924472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:51.924961 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:51.925106 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:51.944161 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:51.945316 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:52.002501 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1294/2556] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:52.013362 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:52.014754 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:52.070223 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec96645-ec52-4101-93ce-fd6fc42dc2a5 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:52.073803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:52.074399 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:52.074672 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:52.074940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:52.084212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:52.085637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:52.094673 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1263/2557] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:05:52.361030 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:52.362451 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:52.374791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119af85-0ea5-4ec2-b527-c3e6d4a339cf None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:52.380265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:52.380967 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:52.381249 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:52.381520 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] PATH_INFO: `/v3/projects/fd28572195764b14888e504bee5960d2` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:52.382695 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd28572195764b14888e504bee5960d2)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:52.384721 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:52.418658 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:52.431543 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd28572195764b14888e504bee5960d2', 'request_id': 'req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.438921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd28572195764b14888e504bee5960d2', 'request_id': 'req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.445968 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd28572195764b14888e504bee5960d2', 'request_id': 'req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.456046 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] The token cache is being invalidate because project fd28572195764b14888e504bee5960d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:52.456400 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.457021 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.458475 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1295/2558] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:52 2022] DELETE /identity/v3/projects/fd28572195764b14888e504bee5960d2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:52.462629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:52.472220 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bb88c2-a886-4f3b-ae98-6b47ac258c8b service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:52.528345 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:52.528975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:52.529255 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:52.529539 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] PATH_INFO: `/v3/users/2d3cf95b93ec49b78fe0c2cda20076e6` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:52.530663 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d3cf95b93ec49b78fe0c2cda20076e6)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:52.532595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:52.577797 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d3cf95b93ec49b78fe0c2cda20076e6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.578434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d3cf95b93ec49b78fe0c2cda20076e6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.585218 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d3cf95b93ec49b78fe0c2cda20076e6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.598382 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d3cf95b93ec49b78fe0c2cda20076e6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.599191 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d3cf95b93ec49b78fe0c2cda20076e6'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:52.611456 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1264/2559] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:52 2022] DELETE /identity/v3/users/2d3cf95b93ec49b78fe0c2cda20076e6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:52.879437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:52.880952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:52.895475 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2eadaa-a7f7-4ada-9896-aaa33f006d76 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:52.953578 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:52.954317 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:52.954472 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:52.954745 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] PATH_INFO: `/v3/projects/db74a41ac4094f90b78c550eb423ba2a` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:52.955851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] RBAC: Authorizing `identity:delete_project(project_id=db74a41ac4094f90b78c550eb423ba2a)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:52.957814 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.008848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db74a41ac4094f90b78c550eb423ba2a', 'request_id': 'req-2ae0a8ee-a238-4179-b6cb-b5549d809fed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.015867 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db74a41ac4094f90b78c550eb423ba2a', 'request_id': 'req-2ae0a8ee-a238-4179-b6cb-b5549d809fed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.021959 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db74a41ac4094f90b78c550eb423ba2a', 'request_id': 'req-2ae0a8ee-a238-4179-b6cb-b5549d809fed'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.032116 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] The token cache is being invalidate because project db74a41ac4094f90b78c550eb423ba2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:53.032439 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.033063 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.034533 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1296/2560] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:52 2022] DELETE /identity/v3/projects/db74a41ac4094f90b78c550eb423ba2a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:53.038668 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.040085 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:53.053128 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb019f7-179b-490e-a5b3-111fcc3e9b9d admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:53.057555 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:53.111443 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:53.112073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.112390 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.112666 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] PATH_INFO: `/v3/users/1729ddadad72404fad951ba192b4f80a` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.113780 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1729ddadad72404fad951ba192b4f80a)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:53.115685 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.162368 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1729ddadad72404fad951ba192b4f80a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.163007 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1729ddadad72404fad951ba192b4f80a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.169967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1729ddadad72404fad951ba192b4f80a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.182473 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1729ddadad72404fad951ba192b4f80a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.183105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-1a9ed9e2-1995-4448-9228-65a7aa30174d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1729ddadad72404fad951ba192b4f80a'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.196284 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1265/2561] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:05:53 2022] DELETE /identity/v3/users/1729ddadad72404fad951ba192b4f80a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:53.463102 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.464605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:53.479010 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0a8ee-a238-4179-b6cb-b5549d809fed admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:53.525733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:53.537504 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:53.538132 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.538411 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.538682 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] PATH_INFO: `/v3/projects/d6f9964b36d64f998910993e6ca17c61` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.539788 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f9964b36d64f998910993e6ca17c61)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:53.541727 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.585921 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f9964b36d64f998910993e6ca17c61', 'request_id': 'req-1e3c89aa-35ab-4a3d-8d49-32edaa736702'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.592722 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f9964b36d64f998910993e6ca17c61', 'request_id': 'req-1e3c89aa-35ab-4a3d-8d49-32edaa736702'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.599598 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f9964b36d64f998910993e6ca17c61', 'request_id': 'req-1e3c89aa-35ab-4a3d-8d49-32edaa736702'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.610108 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] The token cache is being invalidate because project d6f9964b36d64f998910993e6ca17c61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:05:53.610405 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.611155 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1e3c89aa-35ab-4a3d-8d49-32edaa736702 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:05:53.614092 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1297/2562] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:05:53 2022] DELETE /identity/v3/projects/d6f9964b36d64f998910993e6ca17c61 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:05:53.621975 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c196-7686-448d-902d-5ab39689813c None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.622226 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c196-7686-448d-902d-5ab39689813c None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.622549 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c196-7686-448d-902d-5ab39689813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.643039 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-26f8c196-7686-448d-902d-5ab39689813c None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:53.644811 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-26f8c196-7686-448d-902d-5ab39689813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:53.701291 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1266/2563] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:05:53.710303 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4e24a-bc43-4957-b7bf-016aa4d49856 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.710600 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4e24a-bc43-4957-b7bf-016aa4d49856 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.710856 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4e24a-bc43-4957-b7bf-016aa4d49856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.731016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4bf4e24a-bc43-4957-b7bf-016aa4d49856 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:53.732193 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4bf4e24a-bc43-4957-b7bf-016aa4d49856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:53.739947 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1298/2564] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:53.744068 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8c196-7686-448d-902d-5ab39689813c None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.754008 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8c196-7686-448d-902d-5ab39689813c None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:53.758527 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:53.759124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.759404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.759679 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.760353 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:05:53.760658 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:53.762241 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.774013 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1267/2565] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:05:53.778266 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4e24a-bc43-4957-b7bf-016aa4d49856 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.787899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4e24a-bc43-4957-b7bf-016aa4d49856 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:53.792609 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:53.793197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.793476 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.793748 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.794369 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:53.796240 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.822962 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1299/2566] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:05:53 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:53.827124 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.837680 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414cea9-8dca-49e4-89af-d1c8f59b5690 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:53.849447 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:05:53.895009 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:53.895607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.895883 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.896154 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.896802 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:53.898712 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.919009 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1268/2567] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:05:53 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:05:53.923484 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.933899 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6db3be-505e-48b6-b8e5-180d6813c37d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:53.938617 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:53.939212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.939499 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.939766 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.940432 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:53.942112 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.951853 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1300/2568] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:05:53.955710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.966002 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d10068-acde-4395-8096-34b04e94c411 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:53.970563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45515b-e8ea-4ffa-ad9f-44a9248c8a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:53.971157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45515b-e8ea-4ffa-ad9f-44a9248c8a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:53.971438 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45515b-e8ea-4ffa-ad9f-44a9248c8a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:53.971711 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45515b-e8ea-4ffa-ad9f-44a9248c8a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:53.972967 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45515b-e8ea-4ffa-ad9f-44a9248c8a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:53.974541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45515b-e8ea-4ffa-ad9f-44a9248c8a9c admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:53.986041 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1269/2569] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:05:53.989953 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:53.999562 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250ed42-ef1e-4958-9438-dfd76d981099 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:54.004943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:54.005585 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:54.005863 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:54.006137 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] PATH_INFO: `/v3/projects/621854cfb16c4a19a9c3b8f38b265dc6/users/c9890ccd20fd47eb88d0e2955d29f20e/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:54.020904 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=621854cfb16c4a19a9c3b8f38b265dc6, user_id=c9890ccd20fd47eb88d0e2955d29f20e, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:54.023525 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:54.035673 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1301/2570] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:05:53 2022] PUT /identity/v3/projects/621854cfb16c4a19a9c3b8f38b265dc6/users/c9890ccd20fd47eb88d0e2955d29f20e/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:05:54.048115 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be519f-e36d-49ea-b5c2-cb1f245822f1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:54.048645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be519f-e36d-49ea-b5c2-cb1f245822f1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:54.048744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4be519f-e36d-49ea-b5c2-cb1f245822f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:54.076211 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-a4be519f-e36d-49ea-b5c2-cb1f245822f1 None None] MFA Rules not processed for user `c9890ccd20fd47eb88d0e2955d29f20e`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:05:54.077439 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-a4be519f-e36d-49ea-b5c2-cb1f245822f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:54.139512 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1270/2571] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:05:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:05:54.151861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:05:54.153560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:05:54.208683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6a4a7-adf3-425f-b296-92fd29481c8b admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:05:54.212292 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55809958-7387-479d-9d17-9d85d4bc151a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:05:54.212908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55809958-7387-479d-9d17-9d85d4bc151a service glance] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:05:54.213187 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55809958-7387-479d-9d17-9d85d4bc151a service glance] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:05:54.213457 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55809958-7387-479d-9d17-9d85d4bc151a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:05:54.221513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55809958-7387-479d-9d17-9d85d4bc151a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:05:54.222965 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55809958-7387-479d-9d17-9d85d4bc151a service glance] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:05:54.231531 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1302/2572] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:05:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:04.136534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:04.227577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:20.167747 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be519f-e36d-49ea-b5c2-cb1f245822f1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:20.227283 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4be519f-e36d-49ea-b5c2-cb1f245822f1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:20.231654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:20.232269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:20.232572 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:20.232840 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] PATH_INFO: `/v3/users/c9890ccd20fd47eb88d0e2955d29f20e` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:20.233950 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9890ccd20fd47eb88d0e2955d29f20e)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:20.235845 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:20.281981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9890ccd20fd47eb88d0e2955d29f20e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.282929 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9890ccd20fd47eb88d0e2955d29f20e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.289791 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9890ccd20fd47eb88d0e2955d29f20e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.301674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9890ccd20fd47eb88d0e2955d29f20e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.302330 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9890ccd20fd47eb88d0e2955d29f20e'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.314528 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1271/2573] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:06:20 2022] DELETE /identity/v3/users/c9890ccd20fd47eb88d0e2955d29f20e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:06:20.319616 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:20.320131 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:20.320239 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:20.339516 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:20.340733 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:20.397074 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1303/2574] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:20.406600 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:20.407988 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:20.465132 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c5845c-bceb-4b60-8b8d-18d300e490dc admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:20.468728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65962bd6-ca55-42d4-ad97-c78f20c614e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:20.469328 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65962bd6-ca55-42d4-ad97-c78f20c614e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:20.469607 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65962bd6-ca55-42d4-ad97-c78f20c614e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:20.469876 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65962bd6-ca55-42d4-ad97-c78f20c614e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:20.478611 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65962bd6-ca55-42d4-ad97-c78f20c614e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:20.480020 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65962bd6-ca55-42d4-ad97-c78f20c614e5 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:20.489288 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1272/2575] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:20.746739 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:20.748203 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:20.760705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2093da-ca36-48f1-870b-18e6b45dd7b9 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:20.766262 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:20.767287 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:20.767680 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:20.767958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] PATH_INFO: `/v3/projects/621854cfb16c4a19a9c3b8f38b265dc6` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:20.769139 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=621854cfb16c4a19a9c3b8f38b265dc6)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:20.772233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:20.824679 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '621854cfb16c4a19a9c3b8f38b265dc6', 'request_id': 'req-77191b05-94b1-4ad7-9886-35a6c632a5fa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.832394 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '621854cfb16c4a19a9c3b8f38b265dc6', 'request_id': 'req-77191b05-94b1-4ad7-9886-35a6c632a5fa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.838732 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '621854cfb16c4a19a9c3b8f38b265dc6', 'request_id': 'req-77191b05-94b1-4ad7-9886-35a6c632a5fa'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.849464 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] The token cache is being invalidate because project 621854cfb16c4a19a9c3b8f38b265dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:06:20.849750 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.850425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-77191b05-94b1-4ad7-9886-35a6c632a5fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:20.851895 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1304/2576] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:06:20 2022] DELETE /identity/v3/projects/621854cfb16c4a19a9c3b8f38b265dc6 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:06:20.857117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15621f1-8621-49a3-af15-bfa8c226e378 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:20.857404 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15621f1-8621-49a3-af15-bfa8c226e378 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:20.857678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15621f1-8621-49a3-af15-bfa8c226e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:20.877251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-e15621f1-8621-49a3-af15-bfa8c226e378 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:20.878432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-e15621f1-8621-49a3-af15-bfa8c226e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:20.896354 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:20.939231 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1273/2577] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:20.946995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c626-b745-4b94-a184-c040106db478 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:20.947274 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c626-b745-4b94-a184-c040106db478 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:20.947542 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3c626-b745-4b94-a184-c040106db478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:20.966560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-30c3c626-b745-4b94-a184-c040106db478 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:20.967729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-30c3c626-b745-4b94-a184-c040106db478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:20.975851 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1305/2578] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:20.979915 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15621f1-8621-49a3-af15-bfa8c226e378 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:20.989337 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15621f1-8621-49a3-af15-bfa8c226e378 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:20.994392 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:20.994979 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:20.995264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:20.995531 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:20.996166 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:06:20.996506 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:20.998077 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.009480 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1274/2579] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:06:21.013506 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3c626-b745-4b94-a184-c040106db478 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.022347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3c626-b745-4b94-a184-c040106db478 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.026851 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.027429 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.027705 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.027972 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.028635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.030492 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.057309 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1306/2580] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:06:21 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:06:21.061594 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.071856 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2dd01-8897-4901-aed3-81269450eea4 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.126356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.126946 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.127264 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.127541 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.128157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.130348 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.151091 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1275/2581] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:06:21 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:06:21.155233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.165761 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fa06d6-fbce-4c94-be7a-a8b3ad7bf4d1 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.170347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.170932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.171210 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.171479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.172113 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.173725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.183967 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1307/2582] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:06:21.187644 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.197720 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67115752-05c7-4a99-be72-e6c6fec76d11 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.202199 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eeefc7-4f39-4e77-a3eb-8795b857e631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.202787 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeefc7-4f39-4e77-a3eb-8795b857e631 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.203064 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeefc7-4f39-4e77-a3eb-8795b857e631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.203334 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eeefc7-4f39-4e77-a3eb-8795b857e631 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.203966 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eeefc7-4f39-4e77-a3eb-8795b857e631 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.205568 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eeefc7-4f39-4e77-a3eb-8795b857e631 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.216232 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1276/2583] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:06:21.221146 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.231513 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dad523-0f89-43d3-a3a2-d23865949107 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.236096 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.236787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.237069 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.237347 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] PATH_INFO: `/v3/projects/a8a94fe9c7404ea9b2c67469dcaeeac3/users/2e8aa796837d41fa9d1f342a2f8e1c5d/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.254018 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a94fe9c7404ea9b2c67469dcaeeac3, user_id=2e8aa796837d41fa9d1f342a2f8e1c5d, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.256720 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.270276 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1308/2584] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:06:21 2022] PUT /identity/v3/projects/a8a94fe9c7404ea9b2c67469dcaeeac3/users/2e8aa796837d41fa9d1f342a2f8e1c5d/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:06:21.336491 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02120fbd-478b-4784-9173-83de2e78399d None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.336770 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02120fbd-478b-4784-9173-83de2e78399d None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.337042 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02120fbd-478b-4784-9173-83de2e78399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.365106 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-02120fbd-478b-4784-9173-83de2e78399d None None] MFA Rules not processed for user `2e8aa796837d41fa9d1f342a2f8e1c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:21.366293 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-02120fbd-478b-4784-9173-83de2e78399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:21.428820 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1277/2585] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:21.439826 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.441611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:21.463703 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:21.499533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb0bde6-5bec-4cba-895f-2c00222f9a5e admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.503128 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe308d-8c46-4542-8eca-72da89047470 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9075ccb9e3cd46cc9e77caa4060e3c4e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.503983 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe308d-8c46-4542-8eca-72da89047470 service cinder] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.504254 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe308d-8c46-4542-8eca-72da89047470 service cinder] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.504551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe308d-8c46-4542-8eca-72da89047470 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.512567 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe308d-8c46-4542-8eca-72da89047470 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.513982 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe308d-8c46-4542-8eca-72da89047470 service cinder] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.523287 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1309/2586] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:21.547881 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02120fbd-478b-4784-9173-83de2e78399d None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.566582 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:21.612650 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02120fbd-478b-4784-9173-83de2e78399d None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.617105 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.617722 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.617994 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.618261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] PATH_INFO: `/v3/users/2e8aa796837d41fa9d1f342a2f8e1c5d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.619367 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e8aa796837d41fa9d1f342a2f8e1c5d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.621284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.666442 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e8aa796837d41fa9d1f342a2f8e1c5d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:21.667093 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e8aa796837d41fa9d1f342a2f8e1c5d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:21.673936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e8aa796837d41fa9d1f342a2f8e1c5d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:21.685304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e8aa796837d41fa9d1f342a2f8e1c5d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:21.685934 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e8aa796837d41fa9d1f342a2f8e1c5d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:21.698666 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1278/2587] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:06:21 2022] DELETE /identity/v3/users/2e8aa796837d41fa9d1f342a2f8e1c5d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:06:21.703098 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.703384 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.703656 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.723122 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:21.724245 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:21.781479 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1310/2588] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:21.790855 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:21.792261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:21.849374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38681bc4-b462-41c7-b72e-367f50ef5700 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:21.853023 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b6ce-0adb-424a-bb64-1563ef4e2d07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:21.853649 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6ce-0adb-424a-bb64-1563ef4e2d07 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:21.853927 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6ce-0adb-424a-bb64-1563ef4e2d07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:21.854198 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b6ce-0adb-424a-bb64-1563ef4e2d07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:21.862861 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b6ce-0adb-424a-bb64-1563ef4e2d07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:21.864275 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5b6ce-0adb-424a-bb64-1563ef4e2d07 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:21.873445 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1279/2589] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:22.136780 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.138683 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:22.153418 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0db8bf-2ac7-4f68-acd0-677f54e88a8d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.164832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.165470 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.165749 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.166016 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] PATH_INFO: `/v3/projects/a8a94fe9c7404ea9b2c67469dcaeeac3` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.167129 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a94fe9c7404ea9b2c67469dcaeeac3)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.169054 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.215055 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a94fe9c7404ea9b2c67469dcaeeac3', 'request_id': 'req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:22.222140 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a94fe9c7404ea9b2c67469dcaeeac3', 'request_id': 'req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:22.231605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a94fe9c7404ea9b2c67469dcaeeac3', 'request_id': 'req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:22.242933 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] The token cache is being invalidate because project a8a94fe9c7404ea9b2c67469dcaeeac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:06:22.243212 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:22.243834 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-dd246e9a-15f6-42b3-8d4f-c778d4bea9c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:06:22.245322 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1311/2590] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:06:22 2022] DELETE /identity/v3/projects/a8a94fe9c7404ea9b2c67469dcaeeac3 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:06:22.251271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47132-086f-4a3b-b158-23ff9b32b57b None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.251727 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47132-086f-4a3b-b158-23ff9b32b57b None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.251912 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b47132-086f-4a3b-b158-23ff9b32b57b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.272584 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-f4b47132-086f-4a3b-b158-23ff9b32b57b None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:22.273703 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-f4b47132-086f-4a3b-b158-23ff9b32b57b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:22.306560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:22.334240 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1280/2591] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:22.342126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88977-e0cd-445c-ae84-b43ca82bade5 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.342412 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88977-e0cd-445c-ae84-b43ca82bade5 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.342686 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc88977-e0cd-445c-ae84-b43ca82bade5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.365197 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-6bc88977-e0cd-445c-ae84-b43ca82bade5 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:22.366404 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6bc88977-e0cd-445c-ae84-b43ca82bade5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:22.374295 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1312/2592] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:22.378484 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b47132-086f-4a3b-b158-23ff9b32b57b None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.388396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b47132-086f-4a3b-b158-23ff9b32b57b None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.392851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.393454 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.393728 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.393995 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] PATH_INFO: `/v3/domains` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.394629 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81371) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:06:22.394925 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.397073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.408736 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1281/2593] 10.222.0.39 () {58 vars in 1153 bytes} [Sun Oct 23 01:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:06:22.413211 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc88977-e0cd-445c-ae84-b43ca82bade5 None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.422233 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc88977-e0cd-445c-ae84-b43ca82bade5 None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.426907 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.427495 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.427772 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.428044 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.428693 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.430577 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.456788 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1313/2594] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:06:22 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:06:22.461396 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.471317 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc3e898-9b2f-417a-a135-7fbbfb6d8966 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.526065 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.526637 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.526922 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.527189 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.527807 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.529730 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.549765 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1282/2595] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:06:22 2022] POST /identity/v3/users => generated 293 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:06:22.553960 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.563990 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc61c11e-ce89-4599-9265-f230e0e72333 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.569049 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.569631 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.569909 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.570182 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.570815 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.572437 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.582058 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1314/2596] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:06:22.586393 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.595928 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbcb69a-25dd-4ca1-820f-fab5db6996ec admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.601901 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be231665-27d5-46c2-8cb6-a7a8603e9710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.602482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be231665-27d5-46c2-8cb6-a7a8603e9710 admin admin] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.602777 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be231665-27d5-46c2-8cb6-a7a8603e9710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.603031 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be231665-27d5-46c2-8cb6-a7a8603e9710 admin admin] PATH_INFO: `/v3/roles` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.603656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be231665-27d5-46c2-8cb6-a7a8603e9710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.605251 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be231665-27d5-46c2-8cb6-a7a8603e9710 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.616341 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1283/2597] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:06:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:06:22.620224 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.630201 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223160d-6917-4e97-abaf-5cbc7a49267c admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.635208 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.635843 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] REQUEST_METHOD: `PUT` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.636123 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.636425 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] PATH_INFO: `/v3/projects/556e35a126cf4cdfa08a92edd6d58807/users/ec6272e0a4a142749419bc35a2e847ec/roles/f6e93371f84c47b0b0ecf09d4467080d` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.638611 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:22.654510 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] RBAC: Authorizing `identity:create_grant(project_id=556e35a126cf4cdfa08a92edd6d58807, user_id=ec6272e0a4a142749419bc35a2e847ec, role_id=f6e93371f84c47b0b0ecf09d4467080d)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.657216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.670185 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1315/2598] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:06:22 2022] PUT /identity/v3/projects/556e35a126cf4cdfa08a92edd6d58807/users/ec6272e0a4a142749419bc35a2e847ec/roles/f6e93371f84c47b0b0ecf09d4467080d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:06:22.675398 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eef82-151a-47bc-b192-fee186953558 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.675684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eef82-151a-47bc-b192-fee186953558 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.675957 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602eef82-151a-47bc-b192-fee186953558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.695323 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-602eef82-151a-47bc-b192-fee186953558 None None] MFA Rules not processed for user `c668a27e9bfc403386287995d4c4bda0`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:22.696560 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-602eef82-151a-47bc-b192-fee186953558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:22.753501 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1284/2599] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:22.763752 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:22.765444 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:22.821858 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f2fee-7409-4252-92cc-09e791642498 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:22.825447 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:22.826053 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:22.826333 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:22.826605 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:22.835548 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:22.837011 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:22.846247 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1316/2600] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:26.959114 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b6989-61c8-4327-b467-af4b9b9f05a1 None None] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:26.959595 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b6989-61c8-4327-b467-af4b9b9f05a1 None None] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:26.959710 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b6989-61c8-4327-b467-af4b9b9f05a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:26.996110 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.auth.core [None req-379b6989-61c8-4327-b467-af4b9b9f05a1 None None] MFA Rules not processed for user `ec6272e0a4a142749419bc35a2e847ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81371) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:26.997375 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-379b6989-61c8-4327-b467-af4b9b9f05a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:27.032852 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:27.066913 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1285/2601] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 01:06:27.071935 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:27.082505 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c314f-0af1-4a98-9543-1a01646fb107 service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:27.087765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:27.088508 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:27.088853 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:27.089230 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:27.089943 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:27.091927 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:27.123585 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1317/2602] 10.222.0.39 () {60 vars in 1153 bytes} [Sun Oct 23 01:06:27 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:06:27.124803 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b6989-61c8-4327-b467-af4b9b9f05a1 None None] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:27.136590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b6989-61c8-4327-b467-af4b9b9f05a1 None None] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:27.195578 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:27.196194 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] REQUEST_METHOD: `POST` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:27.196504 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:27.196774 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] PATH_INFO: `/v3/users` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:27.197403 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:27.199304 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:27.219439 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1286/2603] 10.222.0.39 () {60 vars in 1144 bytes} [Sun Oct 23 01:06:27 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:06:27.223522 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:27.233632 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea4526-9402-471f-baf8-dc62de3e5c8d admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:27.238231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:27.238822 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:27.239100 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:27.239367 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:27.240019 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:27.241635 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:27.252319 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1318/2604] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:06:27.257213 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:27.268494 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e676056d-74ed-443f-ad36-4acfa891fc2a admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:27.273781 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:27.274428 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:27.274702 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:27.274969 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] PATH_INFO: `/v3/projects/9fc8bee4b37a4723a35730abf2ac61ce/users/c0767a687766412b8abdb5b46b661f7d/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:27.290697 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc8bee4b37a4723a35730abf2ac61ce, user_id=c0767a687766412b8abdb5b46b661f7d, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:27.293387 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:27.307469 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1287/2605] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:06:27 2022] PUT /identity/v3/projects/9fc8bee4b37a4723a35730abf2ac61ce/users/c0767a687766412b8abdb5b46b661f7d/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:06:27.311552 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:27.321926 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e08cb-29a3-4e05-9b0e-f42f81dc1ca3 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:27.376697 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f248b0c-ee42-423c-9c32-9e136604ad63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:27.377304 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f248b0c-ee42-423c-9c32-9e136604ad63 admin admin] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:27.377580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f248b0c-ee42-423c-9c32-9e136604ad63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:27.377848 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f248b0c-ee42-423c-9c32-9e136604ad63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:27.378502 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f248b0c-ee42-423c-9c32-9e136604ad63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:27.380407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f248b0c-ee42-423c-9c32-9e136604ad63 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:27.390979 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1319/2606] 10.222.0.39 () {58 vars in 1122 bytes} [Sun Oct 23 01:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:06:27.395117 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:27.405366 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6605f-1ad5-463f-a558-9d569de72119 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:27.409822 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:27.410468 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:27.410744 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:27.411012 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] PATH_INFO: `/v3/projects/9fc8bee4b37a4723a35730abf2ac61ce/users/c0767a687766412b8abdb5b46b661f7d/roles/4ea11f7df83546718222b4c38fd2c9f2` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:27.412938 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fc8bee4b37a4723a35730abf2ac61ce, user_id=c0767a687766412b8abdb5b46b661f7d, role_id=4ea11f7df83546718222b4c38fd2c9f2)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:27.415519 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:27.425761 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1288/2607] 10.222.0.39 () {60 vars in 1483 bytes} [Sun Oct 23 01:06:27 2022] PUT /identity/v3/projects/9fc8bee4b37a4723a35730abf2ac61ce/users/c0767a687766412b8abdb5b46b661f7d/roles/4ea11f7df83546718222b4c38fd2c9f2 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:06:31.773142 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf791f0-71ac-481d-86ea-7628107fd3ed None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:31.774533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf791f0-71ac-481d-86ea-7628107fd3ed None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:31.774970 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf791f0-71ac-481d-86ea-7628107fd3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:31.817778 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-cdf791f0-71ac-481d-86ea-7628107fd3ed None None] MFA Rules not processed for user `c0767a687766412b8abdb5b46b661f7d`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:31.819397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cdf791f0-71ac-481d-86ea-7628107fd3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:31.874718 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:31.910538 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1320/2608] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:31.925575 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:31.927451 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:32.011579 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa43d56-bb8c-47f0-b03f-452b131a84ea admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:32.015224 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:32.015851 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:32.016127 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:32.016432 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:32.061466 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:32.081297 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:32.082776 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:32.100469 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1289/2609] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:32.262931 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a46c-53a7-4271-bc7e-162245535b25 None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:32.263216 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a46c-53a7-4271-bc7e-162245535b25 None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:32.263519 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5a46c-53a7-4271-bc7e-162245535b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:32.315847 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4bc5a46c-53a7-4271-bc7e-162245535b25 None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:32.317097 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4bc5a46c-53a7-4271-bc7e-162245535b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:32.383389 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1321/2610] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 01:06:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:32.431013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:32.432471 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:32.526181 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8270f-2484-4cfb-9b48-26a1fb2365c2 service glance] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:32.532590 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:32.533203 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:32.533482 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:32.533808 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:32.543149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:32.545134 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:32.555417 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1290/2611] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:06:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:06:41.185289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:41.185791 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:41.185892 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:41.204857 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] MFA Rules not processed for user `ec6272e0a4a142749419bc35a2e847ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:41.206042 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:41.213338 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1322/2612] 10.222.0.39 () {58 vars in 958 bytes} [Sun Oct 23 01:06:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:41.252461 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:41.262674 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d2109-ded7-4198-bc13-e0e4ae2eb6b1 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:41.266233 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b5429182e2fa4e2aa485fc084e32aafd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:41.266821 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:41.267099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:41.267371 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:41.269119 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:41.338817 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:41.340229 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:41.351334 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1291/2613] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:41.577349 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:41.578738 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:41.636355 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71921f87-4898-41b6-bb13-421fb83774ca None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:41.639939 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:41.640565 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:41.640846 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:41.641117 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:41.642832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:41.652126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:41.653581 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:41.661535 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1323/2614] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:41.981478 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:41.982970 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:41.994421 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:42.003280 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419484f4-999e-4588-aaa9-08753eee685e service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:42.007861 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548bf88-5f8b-4ab5-a09c-f3a02bdd6b92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:42.008521 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548bf88-5f8b-4ab5-a09c-f3a02bdd6b92 service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:42.008801 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548bf88-5f8b-4ab5-a09c-f3a02bdd6b92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:42.009075 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548bf88-5f8b-4ab5-a09c-f3a02bdd6b92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:42.010835 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-6548bf88-5f8b-4ab5-a09c-f3a02bdd6b92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:42.028158 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548bf88-5f8b-4ab5-a09c-f3a02bdd6b92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:42.030045 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548bf88-5f8b-4ab5-a09c-f3a02bdd6b92 service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:42.030866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=100 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:42.043253 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1324/2615] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:06:42.061374 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b52f8-aaa4-4b66-b1fc-f0e820a96ed7 service glance] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:42.066586 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:42.068284 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:42.068713 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:42.068981 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:42.070687 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:42.082810 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:42.085045 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:42.096910 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1292/2616] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:06:43.278101 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b4cf0-8f02-4f3b-8e5a-8b9d4054f97d None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:43.278580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b4cf0-8f02-4f3b-8e5a-8b9d4054f97d None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:43.278991 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b4cf0-8f02-4f3b-8e5a-8b9d4054f97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:43.298524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-4a3b4cf0-8f02-4f3b-8e5a-8b9d4054f97d None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:06:43.299751 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-4a3b4cf0-8f02-4f3b-8e5a-8b9d4054f97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:43.307448 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1325/2617] 10.222.0.39 () {58 vars in 1009 bytes} [Sun Oct 23 01:06:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:06:43.321939 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:43.333214 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0f408-fb47-42ca-ab0a-5ad9bc72bbaa service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:43.338172 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:43.338779 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:43.339061 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:43.339333 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:43.347420 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:43.348956 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:43.357126 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1293/2618] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:06:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:06:53.293452 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=90 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:53.353647 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=21 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:06:54.573397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b4cf0-8f02-4f3b-8e5a-8b9d4054f97d None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:54.582787 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b4cf0-8f02-4f3b-8e5a-8b9d4054f97d None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:54.586407 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:54.586996 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:54.587268 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:54.587533 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:54.589258 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:54.598755 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:54.600167 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:54.608271 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1326/2619] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 01:06:55.235551 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:06:55.237256 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:55.253553 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eade7f-5b21-4b82-802d-326519f443ba service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:06:55.259435 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:06:55.260044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:06:55.260347 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:06:55.260627 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:06:55.262356 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:06:55.275866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:06:55.277523 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:06:55.289345 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1294/2620] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:06:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:07:00.820012 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:07:00.835973 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1494d70-2f09-44d7-8b5d-26f87c1c3eee service neutron] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:07:00.848832 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:07:00.849446 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:07:00.849725 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:07:00.849997 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:07:00.851712 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:07:00.864479 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:07:00.866048 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:07:00.876599 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1327/2621] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:07:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:07:05.283893 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:07:07.958161 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:07:07.967381 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38741733-716b-4d74-a4ea-10b85f5a123d service nova] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:07:07.971099 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bb5f82d3cfc14eacbe4cc40bcd0dfb20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:07:07.971717 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:07:07.972019 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:07:07.972288 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:07:07.981162 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:07:07.982563 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:07:07.992318 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1295/2622] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:07:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:07:10.871995 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=12 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:07:11.255111 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:07:11.268958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19779c47-113b-4e2d-a492-c4ce2d83988c service nova] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:07:11.276375 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec83b09-414e-40cd-ad96-189e0872811f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f04fb9e1d1bd42ac815078b61f6a17f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:07:11.277127 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec83b09-414e-40cd-ad96-189e0872811f service nova] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:07:11.277586 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec83b09-414e-40cd-ad96-189e0872811f service nova] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:07:11.277955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec83b09-414e-40cd-ad96-189e0872811f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:07:11.279932 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cec83b09-414e-40cd-ad96-189e0872811f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:07:11.297507 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec83b09-414e-40cd-ad96-189e0872811f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:07:11.299202 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec83b09-414e-40cd-ad96-189e0872811f service nova] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:07:11.312006 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1328/2623] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:07:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:07:17.987686 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:07:21.306955 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=6 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:08:30.609149 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:08:30.610706 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:08:30.625656 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b8e35a-22eb-489c-be10-321daebed037 service neutron] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:08:30.630571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:08:30.631427 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:08:30.631705 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:08:30.631977 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:08:30.633645 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:08:30.644786 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:08:30.646346 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:08:30.656930 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1296/2624] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:08:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:08:40.652542 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=6 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:09:45.825839 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc03c2-3109-47d4-9f24-975922709b3f None None] REQUEST_METHOD: `POST` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:09:45.826330 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc03c2-3109-47d4-9f24-975922709b3f None None] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:09:45.826544 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc03c2-3109-47d4-9f24-975922709b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:09:45.846126 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.auth.core [None req-babc03c2-3109-47d4-9f24-975922709b3f None None] MFA Rules not processed for user `232811b7770447b6a12f4a806a7b0119`. Rule list: `[]` (Enabled: `True`). {{(pid=81372) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:09:45.847289 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-babc03c2-3109-47d4-9f24-975922709b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:09:45.854268 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1329/2625] 10.222.0.39 () {58 vars in 967 bytes} [Sun Oct 23 01:09:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 01:09:45.895160 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:09:45.906936 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20666109-3755-4df1-a655-c14301fa2eaa service placement] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:09:45.912678 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0800a1f4140a4d098eadb94eeba922be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:09:45.913287 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:09:45.913571 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:09:45.913843 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:09:45.922932 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:09:45.924534 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:09:45.932669 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1297/2626] 10.222.0.39 () {60 vars in 1384 bytes} [Sun Oct 23 01:09:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:09:49.317952 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc03c2-3109-47d4-9f24-975922709b3f None None] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:09:49.331198 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc03c2-3109-47d4-9f24-975922709b3f None None] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:09:49.336204 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:09:49.336841 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] REQUEST_METHOD: `GET` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:09:49.337118 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:09:49.337388 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:09:49.339293 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:09:49.351084 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:09:49.353231 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:09:49.365414 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1330/2627] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:09:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:09:52.479809 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:09:52.549653 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221ff6c-87d5-4a23-b232-6bced4c59dd7 service swift] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:09:52.554129 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:09:52.554762 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:09:52.555036 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:09:52.555307 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] PATH_INFO: `/v3/users/ec6272e0a4a142749419bc35a2e847ec` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:09:52.557013 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec6272e0a4a142749419bc35a2e847ec)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:09:52.558917 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:09:52.607434 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec6272e0a4a142749419bc35a2e847ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.608033 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec6272e0a4a142749419bc35a2e847ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.615269 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec6272e0a4a142749419bc35a2e847ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.628212 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec6272e0a4a142749419bc35a2e847ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.628940 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec6272e0a4a142749419bc35a2e847ec'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.641479 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1298/2628] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:09:52 2022] DELETE /identity/v3/users/ec6272e0a4a142749419bc35a2e847ec => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:09:52.810390 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:09:52.821765 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d57a0-c068-4630-a8f1-a96f71134f38 service placement] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:09:52.887397 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:09:52.887958 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:09:52.888237 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:09:52.888541 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] PATH_INFO: `/v3/projects/556e35a126cf4cdfa08a92edd6d58807` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:09:52.889650 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=556e35a126cf4cdfa08a92edd6d58807)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:09:52.891576 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:09:52.938153 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '556e35a126cf4cdfa08a92edd6d58807', 'request_id': 'req-cab3af39-527b-4636-9ccc-3b6fbd220d98'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.945493 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '556e35a126cf4cdfa08a92edd6d58807', 'request_id': 'req-cab3af39-527b-4636-9ccc-3b6fbd220d98'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.952455 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '556e35a126cf4cdfa08a92edd6d58807', 'request_id': 'req-cab3af39-527b-4636-9ccc-3b6fbd220d98'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.962560 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] The token cache is being invalidate because project 556e35a126cf4cdfa08a92edd6d58807 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:09:52.962850 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.963511 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:52.964994 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1331/2629] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:09:52 2022] DELETE /identity/v3/projects/556e35a126cf4cdfa08a92edd6d58807 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:09:52.969062 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:09:52.970540 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:09:52.983982 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fcf177-0d0a-4c65-9368-0c2521a5ed21 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:09:53.038082 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:09:53.038684 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:09:53.039017 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:09:53.039384 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] PATH_INFO: `/v3/users/c0767a687766412b8abdb5b46b661f7d` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:09:53.041044 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0767a687766412b8abdb5b46b661f7d)` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:09:53.042952 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:09:53.048073 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:09:53.087826 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0767a687766412b8abdb5b46b661f7d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.088495 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0767a687766412b8abdb5b46b661f7d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.095205 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0767a687766412b8abdb5b46b661f7d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.107207 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0767a687766412b8abdb5b46b661f7d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.107830 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.notifications [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0767a687766412b8abdb5b46b661f7d'} {{(pid=81371) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.120265 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1299/2630] 10.222.0.39 () {58 vars in 1224 bytes} [Sun Oct 23 01:09:52 2022] DELETE /identity/v3/users/c0767a687766412b8abdb5b46b661f7d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:09:53.290551 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Authenticating user token {{(pid=81372) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:09:53.292059 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.fernet_utils [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81372) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:09:53.307872 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab3af39-527b-4636-9ccc-3b6fbd220d98 admin admin] Validating token access rules against request {{(pid=81372) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:09:53.366729 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c668a27e9bfc403386287995d4c4bda0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ee25af34c3554966b7a70b6f9a37af9a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81372) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:09:53.367359 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:09:53.367637 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] SCRIPT_NAME: `/identity` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:09:53.367908 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] PATH_INFO: `/v3/projects/9fc8bee4b37a4723a35730abf2ac61ce` {{(pid=81372) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:09:53.369626 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fc8bee4b37a4723a35730abf2ac61ce)` {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:09:53.371524 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] RBAC: Authorization granted {{(pid=81372) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:09:53.382296 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:09:53.417833 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fc8bee4b37a4723a35730abf2ac61ce', 'request_id': 'req-1824884f-0b01-4da2-bd11-a140f936a254'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.424066 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fc8bee4b37a4723a35730abf2ac61ce', 'request_id': 'req-1824884f-0b01-4da2-bd11-a140f936a254'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.430083 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fc8bee4b37a4723a35730abf2ac61ce', 'request_id': 'req-1824884f-0b01-4da2-bd11-a140f936a254'} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.440265 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] The token cache is being invalidate because project 9fc8bee4b37a4723a35730abf2ac61ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81372) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 01:09:53.440580 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.441209 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG keystone.notifications [None req-1824884f-0b01-4da2-bd11-a140f936a254 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81372) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 01:09:53.442662 dv-cinder-master-3-smb devstack@keystone.service[81372]: [pid: 81372|app: 0|req: 1332/2631] 10.222.0.39 () {58 vars in 1233 bytes} [Sun Oct 23 01:09:53 2022] DELETE /identity/v3/projects/9fc8bee4b37a4723a35730abf2ac61ce => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:10:03.116271 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:10:03.437376 dv-cinder-master-3-smb devstack@keystone.service[81372]: DEBUG dbcounter [-] [81372] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81372) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:13:33.607046 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Authenticating user token {{(pid=81371) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:13:33.608464 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:13:33.747280 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ebdc1a-29ba-419d-8868-d8e8c8b49458 admin admin] Validating token access rules against request {{(pid=81371) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:13:33.805261 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70cc91d-5ed5-4adc-9ac2-4ed5dc8d98d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '145f39f828664c56903ae5aeffa4dd52', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b5fa3bce9d5a404bb7352ae8867cf4ab', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81371) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:13:33.805866 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc91d-5ed5-4adc-9ac2-4ed5dc8d98d2 service placement] REQUEST_METHOD: `GET` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:13:33.806147 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc91d-5ed5-4adc-9ac2-4ed5dc8d98d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:13:33.806423 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70cc91d-5ed5-4adc-9ac2-4ed5dc8d98d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81371) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:13:33.808157 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.fernet_utils [None req-d70cc91d-5ed5-4adc-9ac2-4ed5dc8d98d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81371) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:13:33.819990 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cc91d-5ed5-4adc-9ac2-4ed5dc8d98d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:13:33.821654 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70cc91d-5ed5-4adc-9ac2-4ed5dc8d98d2 service placement] RBAC: Authorization granted {{(pid=81371) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:13:33.831608 dv-cinder-master-3-smb devstack@keystone.service[81371]: [pid: 81371|app: 0|req: 1300/2632] 10.222.0.39 () {60 vars in 1365 bytes} [Sun Oct 23 01:13:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 01:13:43.827236 dv-cinder-master-3-smb devstack@keystone.service[81371]: DEBUG dbcounter [-] [81371] Writing DB stats keystone:SELECT=89 {{(pid=81371) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}